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1  Introduction 


The  Air  Force  contract  on  Graphic  Systems  Representation  has  three  major  phases: 
Analysis,  Recommendation,  and  Final  Demonstration.  Figure  1-1  depicts  the  major  tasks  to 
accomplish  each  of  these  phases. 

The  Analysis  phase  consists  of: 

1.  The  investigation  and  analysis  of  the  latest  computer  graphics  technology  which  can 
potentially  have  a  significant  benefit  in  the  display  of  Air  Force  tactical  information. 
This  technology  review  includes  graphics  algorithms,  computer  hardware,  computer 
software,  and  design  methodologies.  The  technology  can  either  exist  today  or  be 
available  within  the  time  frame  of  about  five  years. 

2.  The  investigation  of  Air  Force  tactical  planning  applications  with  particular  emphasis 
on  the  analysis  of  existing  graphical  techniques  used  in  thi-fse  applications.  The  primaiy 
source  of  application  knowledge  has  been  a  combination  of  consultants  (e.g,  former  Air 
Force  pilots),  demonstrations  of  existing  computerized  technical  planning  systems  at 
Rome  Air  Development  Center,  video  tapes  of  tactical  planning  systems,  and 
reports/manuals  describing  existing  tactical  planning  sy.stems. 

A  major  output  of  this  Analysis  phase  is  this  document.  Graphical  Requirements  for  Air 
Force  Tactical  Decision  Aids.  The  purposes  of  this  report  are  to  describe  the  graphical  needs  or 
requirements  assxiated  with  Air  Force  tactical  planning,  and  to  describe  2D  and  3D  graphical 
techniques  that  address  these  requirements.  The  description  of  requirements  and  potential 
graphical  techniques  are  divided  into  six  major  categories: 

Section  2  Planning 

Section  3  Maps 

Section  4  Threat  Zones 

Section  5  Weather 

Section  6  Route  Planning 

Section  7  User  Interaction 

In  Section  2,  the  high-level  graphics  requirements  of  the  planning  process  are  investigated. 
The  next  four  sections  -  Maps.  Threat  Zones.  Weather,  and  Route  Planning  -  were  found  to  be 
pervasive  throughout  all  the  planning  phases,  and  are  addressed  separately.  In  Section  7,  user 
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Successive  Rapid  Prototyping 


interaction  is  singled  out  because  it  is  a  critical  element  in  the  acceptance  and  implementation  of 
any  new  ideas,  including  new  approaches  in  graphic  systems  representation.  A  summary  of  the 
graphical  requirements  and  potential  graphics  solutions  is  contained  at  the  end  of  each  section. 

A  second  major  reason  for  this  report  is  to  focus  attention  on  the  next  part  of  this  contract, 
the  Recommendation  phase.  This  document  identifies  many  graphics  needs  and  new  techniques 
that  can  be  of  potential  benefit.  One  of  the  first  steps  in  the  Recommendation  phase  is  to  begin 
to  “screen"  the  large  set  of  applications,  graphics  needs,  and  potential  technologies  and  to  select 
“graphics  ideas”  for  rapid  prototype  development.  The  feedback  from  the  Air  Force  is  essential 
in  selecting  and  prioritizing  the  most  important  graphics  requirements.  At  the  end  of  the  rapid 
prototyping  phase,  we  will  jointly  with  the  Air  Force  select  the  final  demonstration(s)  and  video 
tape  to  be  developed  and  delivered. 
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2  Planning 


2.1  Introduction 

Mission  planning  is  the  fundamental  process  of  the  tactical  Air  Force  environment. 
Efficient  planning  requires  the  assimilation  of  large  amounts  of  information,  along  with  the 
capability  to  evaluate  the  plan  at  any  time.  In  addition,  since  deviation  from  plan  is  a  likely 
occurrence  in  the  tactical  Air  Force  environment,  it  is  important  to  provide  methods  for 
monitoring  and  reacting  to  such  a  situation.  Indeed,  the  planning  process  is  inherently  dynamic, 
and  the  single  most  important  aspect  to  successful  planning  is  the  ability  to  rapidly  interact  with 
the  process. 

Computers  provide  a  natural  tool  for  the  planning  process.  Their  basic  function  has  been  to 
manage  data,  assisting  decision  makers  as  they  assimilate  mission  information.  Computers  also 
provide  an  evaluation  mechanism,  using  rapid  calculation  to  provide  feedback  to  the  planner. 
Unfortunately,  computers  often  generate  so  much  information,  so  quickly,  that  the  planning 
process  is  swamped  with  detail.  The  results  are  then  that  much  of  the  information  is  either  lost  or 
ignored,  or  the  relative  importance  of  information  is  lost  to  the  planner.  Therefore,  computer 
graphics  are  useful  in  the  planning  process  for  a  number  of  reasons: 

•  Data  Simplification  Reduce  large  amounts  of  data,  cr  complex  data  inter-relationships 
to  simpler,  easier  to  understand  representations.  For  example,  a  table  of  numbers  can 
be  represented  as  a  bar  chart  or  2D  plot. 

•  Data  Access  Provide  access  to  data  through  straightforward,  intuitive  interactions.  For 
example,  clicking  on  an  icon  with  a  mouse  to  display  characteristics. 

•  Enhanced  Cognition  Design  displays  to  increase  the  transfer  of  important  information 
to  the  planner.  For  example,  use  color  contour  shading  to  indicate  threat  representation 
at  different  locations  on  a  map. 

•  Focus  Computer  graphics  can  provide  focus  to  an  activi^.  For  example,  the  use  of  sound 
or  color  is  often  used  to  draw  attention  to  a  particular  area. 

In  this  section  high-level  graphics  requirements  for  the  planning  process  are  developed. 
Many  of  the  details  of  the  planning  process  (such  as  threat  representation)  are  covered  in  later 
sections.  To  develop  these  requirements  we  will  consider  the  planning  process  of  consisting  of 
four  parts:  mission  preparation,  simulation,  execution,  and  review. 
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2.2  Mission  Preparation 

In  mission  preparation  the  task  is  to  understand  the  goal(s)  of  the  mission,  as  well  as  to 
develop  initial  approaches  to  achieving  the  goal.  This  process  primarily  involves  data 
acquisition;  the  data  being  current  resources,  constraints,  target,  intelligence,  and  prior  history. 
In  general,  because  of  the  scale  of  the  problem,  most  of  the  graphics  for  mission  preparation 
involve  2D  graphics.  It  is  only  when  needing  detailed  knowledge  of  terrain  or  target  that  3D 
graphics  are  required. 

The  graphical  requirements  for  mission  preparation  include: 

•  Maps  serve  as  the  foundation  for  understanding  a  tactical  situation,  especially  when 
augmented  with  supplemental  information  such  as  threats,  transit  routes,  tanker  routes, 
front  line  of  troops,  political  and  geographic  boundaries  and  so  on.  However,  to  be 
useful  and  avoid  information  overload,  the  map  representations  must  be  quite  flexible 
so  that  inquiries  into  any  data  on  the  map  provide  rapid  and  effective  information 
transfer.  For  example,  pointing  at  an  iconic  representation  of  a  threat  on  the  map  yields 
pertinent  information  about  that  threat. 

•  Map  Overlays  prevent  information  overload  by  logically  grouping  data,  allowing  the 
planner  to  interact  with  only  selected  information.  One  example  is  to  group  roads, 
thereby  allowing  the  user  to  manipulate  all  roads  simultaneously.  Typically  this  would 
be  useful  to  turn  the  visibility  of  the  roads  either  on  or  off,  to  avoid  cluttering  a  display. 
The  grouping  mechanism  needs  to  be  general,  so  that  the  planner  can  decide  what  is 
appropriate  for  his  application. 

•  Spreadsheets  &  Forms  are  useful  tools  for  summerizing  numerical  data,  as  well  as 
entering  or  operating  on  numerical  data.  It  is  also  possible  to  create  graphical 
spreadsheets  and  forms,  so  that  spreadsheets  ceils  can  contain  graphical  images. 
Resources  can  also  be  depicted  on  graphical  forms,  as  for  example  allocating  ordnance 
to  aircraft.  In  this  case,  the  aircraft  of  choice  could  be  shown  along  with  possible 
ordnance  combinations,  and  by  interacting  with  the  display  the  planner  could  assign 
it. 

•  2D  Graphics  capability  such  as  standard  business  graphics  is  required.  These 
representations  include  pie  charts,  bar  charts,  x-y  plots,  multi-dimensional  plots, 
statistical  plots,  and  so  forth.  These  provide  a  simple  and  familiar  method  for  resource 
and  information  management.  These  displays  must  be  used  quite  carefully  to  avoid 
information  overload. 

•  2D  Pictorials  such  as  iconic  representations  are  one  of  the  most  useful  graphic 
representations  available.  Carefully  crafted  pictorials  can  represent  an  enormous 
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amount  of  information  in  an  succinct  and  readily  understandable  manner.  The  key  to 
useful  pictorials  is  to  associate  data  directly  to  its  various  components,  and  to  clearly 
represent  each  piece  of  data.  For  example,  the  sensitivity  and  allowable  range  of  a 
parameter  can  be  displayed  by  color  coding  a  slider:  the  slider  indicates  maximum  and 
minimum  range  information,  the  color  of  the  slider  represents  the  sensitivity  at  the 
current  value  of  parameter. 

•  Multiple  Displays  are  mandatory  for  assisting  plaimers  in  assimilating  information. 
Usually  planning  aids  are  written  to  address  just  a  small  set  of  needs,  but  often  other 
information  is  pertinent  to  the  planning  process.  The  planner  needs  to  look  at  multiple 
sources  of  information  simultaneously;  in  addition,  data  in  each  display  needs  to  be 
transferable  from  one  into  another.  Simple  cut  and  paste  paradigms  are  possibilities, 
but  future  data  requirements  demand  that  the  context  of  the  data  is  provided  as  well. 


•  Interactive  Displays  are  vital  to  encourage  the  free  thinking  and  exploration  required 
in  the  planning  process.  Users  of  the  system  must  feel  their  creative  abilities  empowered 
by  the  system  rather  than  stifled  by  it.  Besides  providing  quick  feedback,  displays  must 
be  predictable  to  use,  robust,  provide  guidance,  and  allow  easy  tailoring  so  that  users 
have  control  over  the  information  portrayal. 

•  Customizing  the  layout  and  informational  content  of  a  planning  aide  is  helpful  to  users 
of  computer  systems  because  each  individual  has  their  own  idea  of  what  information 
is  important.  By  customizing  a  graphic  interface,  a  planner  can  improve  the  effective 
knowledge  transfer  rate.  Of  course,  some  users  can  create  interfaces  that  are 
detrimental  to  effective  use,  or  can  develop  interfaces  that  no  one  else  can  use. 

•  3D  Graphics  are  generally  useful  only  in  those  cases  where  the  vertical  dimension  is 
significant  as  compared  to  the  horizontal  dimension.  In  most  planning  operations  this 
is  not  the  case.  (The  following  sections  discuss  some  applications  of  3D  graphics).  3D 
graphics  are  also  useful  for  representing  certain  forms  of  data,  such  as  multi-variate 
statistical  information,  or  complex  icons  or  pictorials.  However,  the  difficulty  and 
ambigui^  of  interacting  with  3D  images  means  that  it  should  be  used  in  a  limited  way 
in  the  higher  levels  of  the  planning  process. 

•  Sound  is  generally  used  as  a  supplemental  channel  for  information  transfer,  especially 
when  communicating  vital  information.  Since  sound  provides  another  channel  for 
information  transfer,  input  or  output  is  available  to  a  planner  who  has  his  other  senses 
are  engaged  with  additional  tasks. 
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•  Animation  is  often  necessaiy  because  the  information  is  of  a  temporal  nature.  In  these 
cases,  developing  an  understanding  of  the  data  can  only  be  developed  by  viewing  it  in 
all  its  dimensions,  including  time.  However,  since  animation  processes  are  often 
difHcult  to  develop,  they  have  to  be  generated  automatically,  or  with  a  small  amount 
of  user  interaction. 

2.3  Mission  Simulation 

In  mission  simulation  the  task  is  to  evaluate  potential  plans,  improving  them  as  necessary  to 
meet  mission  constraints  and  goals.  Here  the  process  is  considered  to  be  iterative;  that  is, 
mission  simulation  and  preparation  proceed  hand  in  hand  until  an  acceptable  solution  is 
determined.  Therefore,  the  graphic  requirements  in  this  phase  are  a  superset  of  those  required 
by  mission  preparation,  with  the  added  requirement  of  simulation. 

Mission  simulation  is  most  effective  when  it  is  as  realistic,  i.e.,  meaning  as  much  like  an 
actual  mission  as  possible.  In  this  sense,  mission  simulation  is  identical  to  mission  execution;  the 
only  difference  being  that  system  is  being  driven  by  underlying  gaming  or  simulation  algorithms. 
Therefore,  the  graphics  requirements  for  mission  simulation  include  the  requirements  for 
mission  execution  as  well.  This  are  discussed  in  the  next  section. 

2.4  Mission  Execution 

As  a  mission  unfolds,  it  is  necessaiy  to  provide  and  update  a  detailed  description  of  the 
current  mission  or  battle  status.  This  description  must  be  as  accurate  as  possible  and  cover  your 
owTi  force,  other  friendly  forces,  and  enemy  forces.  A  wide  variety  of  information  is  used  to 
present  and  evaluate  the  current  status  of  a  mission  or  battle: 

•  Intelligence  reports  on  the  location,  damage,  destruction,  and  movement  of  youi  own 
resources  and  enemy  targets  and  resources 

•  Weather  processes  which  can  affect  original  mission  plans  (e.g^  plane  routings),  target 
visibili^,  intelligence  gathering  (e.g..  satellite  photos),  movement  of  enemy  threats,  and 
resuppfy  of  your  own  or  enemy  resources 

•  Communication  links  for  receiving  intelligence  and  weather  information  and  for 
directing  actions  of  your  own  resources 

•  Historical  knowledge  of  enemy  strate^es  and  tactics  wlien  faced  with  certain  scenarios 

•  Logistics  and  resupply  processes  for  your  own  and  enemy  resources 
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•  Alternate  sources  of  sensor  data  (radar,  IR,  visual)  from  which  a  situation  assessment 
must  be  fused  or  derived 

The  graphical  requirements  for  describing  and  assessing  a  mission  or  battle  situation  as  itis 
executed  and  unfolds  are: 

•  Maps  should  be  available  to  display  the  geographical  area  over  which  the  mission  or 
battle  is  taking  place.  The  maps  are  ideally  in  digital  format  residing  on  a  disc  or  must 
be  scanned  and  converted  into  digital  format.  The  maps  typically  have  to  cover  a  variety 
of  scales  from  thousands  of  miles  to  miles/thousands  of  feet.  Elevation  and  feature  data 
(rivers,  roads,  mountains)  are  essential  MAP  elements. 

•  Map  overlays  can  show  spatial  and  logical  relationships  among  relevant  objects.  One 
such  overlay  is  the  depiction  of  threats  (see  Section  4).  This  overlay  could  show  location, 
range  of  lethality,  and  probability  of  lethality  contours  for  a  single  type  of  threat  or 
multiple  threats.  Transparency  or  choice  of  color  can  be  used  to  emphasize  a  specific 
target  type. 

•  Multiple  Displays  can  be  used  to  maintain  a  simultaneous  perspective  of  different  but 
important  mission  or  battle  information.  Windowing  environments  are  typical  solution 
for  this  requirement.  For  example,  one  window  can  depict  a  large  geographical  scale 
showing  a  2D  representation  of  all  known  targets/threats,  while  other  windows  can  show 
a  3D  representation  of  threats  (lethality  zones)  for  more  detailed  geographical  areas. 


•  Weather  patterns  can  be  critical  in  affecting  your  own  or  enemy  tactics  and  resource 
allocation.  A  hierarchy  of  detail  from  2D  to  3D  displays  are  required.  A  large  scale  map 
overlay  covering  hundreds  to  thousands  of  miles  can  show  a  2D  representation  of 
weather  patterns,  with  symbols  and/or  moving  arrows  showing  such  weather  patterns 
as  snow,  rain,  and  wind.  The  symbol  or  pattern  can  be  depicted  over  a  2D  geographical 
area.  More  sophisticated  3D  displays  can  be  shown  to  depict  and  understand  weather 
patterns  over  a  smaller  geographical  area.  For  example,  pilot  visibility  can  be  shown  as 
a  function  of  elevation.  3D  shaded  images,  isometric  surfaces,  and  transparency  will  be 
important  requirements  for  such  displays. 

•  Communication  links  are  important  for  gathering  intelligence,  finding  out  resources 
status,  and  issuing  orders.  A  2D  display  of  the  available  (and  damaged)  communication 
paths  for  own  or  enemy  forces  can  provide  a  quick  assessment  of  available 
communication  options.  Different  colors,  line  patterns,  or  overlays  can  show  different 
types  of  communication  links. 
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•  Organization  charts  showing  the  relationships  among  your  own  forces  (force,  squadron, 
wing,  unit)  can  be  helpful  in  identifying  available  resources.  Such  charts  may  also  be 
helpful  in  reviewing  enemy  forces.  2D  geometry  (lines,  boxes,  circles)  would  suffice  for 
this  requirement. 

•  For  computer-based  tracking  of  mission  execution,  it  is  essential  to  review  current  battle 
status  and  evaluate  alternative  tactics  during  the  battle  with  interactive  displays.  Large 
screen  displays  (a  few  feet  in  width  and  height)  would  be  extremely  useful  in  depicting 
an  entire  region  or  sub-region  of  battle.  User-friendly  interaction  with  a  small  or  large 
screen  is  important.  The  use  of  a  device  such  as  a  Telestrator  would  be  valuable.  For 
example,  in  football  sports  coverage,  the  announcer  draws  the  explanation  of  a  football 
play  (plan  or  execution)  on  a  tablet  and  it  is  superimposed  on  a  TV  display  screen  in 
real-time.  The  same  concept  can  be  used  in  AF  applications.  Simple  2D  hand-drawn 
graphics  representing  flight  paths,  communication  links,  logistic  routes,  etc.,  can  be 
immediately  shown  on  the  large  screen  display.  Large  screen  flat  displays  would  permit 
a  picture  of  the  battle  scene  more  compactly  and  conveniently  (on  a  table  versus  wall 
screen). 

•  There  is  a  requirement  to  show  and  evaluate  alternate  tactical  plans  during  a  battle 
situation.  Computer  animation  can  show  important  logistical  and  spatial  relationships 
and  must  be  in  real-time.  Contingency  plans  can  be  demonstrated  using  animation, 
such  as  alternate  flight  plans  if  a  friendly  base(s)  is  destroyed.  2D  animation  can  show 
line  paths  of  objects  over  time.  3D  animation  can  show  realistic  color-shaded  objects 
and  their  movement  over  time.  The  animation  can  show  the  expected  (or  other)  enemy 
tactical  responses  to  your  own  tactical  plan  and  your  planned  response. 

•  Storage  of  historical  knowledge  of  enemy  tactics  and  strategies  is  a  requirement  for 
mission  execution,  tracking,  and  resource  allocation.  Graphics  requirements  aie  the 
ability  to  display  these  tactics  and  strategies,  using  2D  geometry  and  2D/3D  animation. 
2D  graphics  icons  could  be  used  as  a  symbology  to  represent  an  overall  enemy  tactic, 
e.g.,  for  user  interface  access  to  a  specific  tactic.  Optical  discs  can  be  a  cost  effective 
method  for  storing  these  tactics. 

•  There  are  many  sources  of  sensor  data,  intelligence  and  other  battle  information  which 
are  not  known  with  certainty.  Graphics  techniques  are  required  to  represent  the 
confidence  level  or  uncertainty  of  this  information.  For  example,  artificial  intelligence 
techniques  can  determine  the  certainty  of  drawing  a  conclusion  from  a  known  set  of 
information  inputs  (each  having  their  own  certainty).  2D  graphical  methods  can  be  used 
to  comprehend  the  certainty  or  confidence  of  a  set  of  sensor  data  or  a  conclusion  derived 
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from  the  application  of  AI  techniques.  For  example,  a  bar  graph  or  the  degree  of  color 
shading  can  show  increasing  (or  decreasing)  certainty, 

•  Automated  warnings  can  be  helpful  in  representing  an  actual  or  planned  tactical  action. 
Intelligence  can  be  built  into  the  battle  tracking  process  such  that  when,  for  example, 
a  communication  link  fails  or  a  certain  level  or  resources  are  damaged  or  destroyed, 
a  warning  occurs.  Warning  representations  can  include  sound  and  computerized  voice 
output.  Graphic  warning  representations  can  include  blinking,  color  change,  or 
combinations  of  these.  For  example,  when  a  specific  route  had  been  flown  a  certain 
number  of  times  to  reach  targets  in  a  certain  geographical  area,  the  route  path  would 
blink  to  warn  the  planner  of  this  fact. 

2.5  Mission  Review 

Mission  review  is  the  analysis  and  understanding  of  the  actions  and  events  that  took  place  in 
a  mission,  in  order  to  learn  and  apply  information  and  techniques  that  can  be  used  to  improve 
future  mission  plans  and  execution.  Relevant  mission  information  that  is  useful  to  review 
includes: 

•  Enemy  tactics  and  movement  to  evade  ovm  forces 

•  Location  and  type  of  enemy  weapons  and  sensors 

•  Terrain  features  that  can  help  in  terrain  masking,  navigation,  and  avoidance  of  trouble 
areas 

•  Enemy  counter-communication  and  sensor-interference  tactics 

Graphical  requirements  for  mission  review  include: 

•  Speed  of  data  input  and  output  display  is  needed  to  convey  and  understand  a  large 
amount  of  infonnaticn  in  s  .short  time  period.  The  time  between  missions  for  review  of 
a  previous  mission  can  be  short  (hours).  Methods  for  quickly  inputing  relevant  data  that 
can  be  shown  on  a  display  include:  on-line  forms  or  tables  that  can  be  quickly  accessed 
via  menus  or  graphical  icons  for  keyboard  entry  or  item  selection;  voice  recognition, 
hand-drawn  graphics  that  can  be  digitally  scanned  or  electronically  captured  (See 
section  2.4,  Telestrator)  for  display;  touch  screens  for  indicating  approximate 
geographical  locations  of  important  features  or  known  targets  and  threats,  and  the 
ability  to  input  or  select  2D  geometry  for  indicating  flight  routes,  turn  points,  and  so 
forth.  The  most  critical  requirements  are  that  data  entry  is  fast  and  convenient. 

•  As  in  Mission  Execution  (Section  1.4)  there  is  the  requirement  to  Interact  with  the  r  eview 
of  the  mission.  Hence,  the  Mission  Review  graphical  requirements  for  map  overlays. 
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windows,  weather  patterns,  organization  charts,  interactive  displays,  animation,  storage 
of  enemy  tactics,  and  warnings  are  fundamentally  the  same  as  Mission  Execution. 

•  Another  important  requirement  is  the  ability  to  compare  the  actual  mission  with  the 
original  mission  plan.  Animation  and  multiple  window  displays  can  be  used  to  satisfy 
this  requirement.  For  example,  one  can  animate  the  original  plan  in  one  window  and 
simultaneously  animate  the  actual  execution  in  a  second  window  -  and  stop  the 
animation  at  specific  times  to  analyze  and  understand  the  reasons  for  the  differences. 
Or  one  can  overlay  the  animations  (such  as  a  plane  routing)  of  the  original  plan  and 
actual  execution  simultaneously  in  a  single  display  to  see  and  understand  the  deviations. 

•  Storage  of  previous  tactical  plans  fyour  own  or  enemy)  is  a  critical  requirement  for 
Mission  Review.  Important  review  questions  are  always  What  mistakes  did  I  make?  and 
How  can  I  do  better?  Graphics  can  permit  ready  access  to  these  plans  via  graphical  icons, 
table  lookup  and  multiple  windows.  Using  animation,  one  can  compare  the  actual  battle 
tactic  v^th  other  stored  alternate  tactics. 

2.6  Summary 

The  following  is  a  brief  summary  of  the  graphics  requirements  for  the  planning  process. 

•  Interactive  Displays 

•  Encourage  exploration  of  alternatives 

•  One  of  the  most  important  features  of  a  useful  system 

•  Large  screen  displays  for  briefing  and  monitoring 

•  Use  Telestrator  to  interact  with  video  displays 

•  Multiple  Displays 

Provide  multiple,  simultaneous  views  of  information  Allow  transfer  of  data  from  one  display 
to  another 

•  Customization 

Allow  the  user  to  adjust  displays  according  to  the  needs  of  the  particular  situation  Can 
enhance  cognition  process 

•  2D/3D  Pictorials  (Other  sections  contain  additional  information) 
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•  Simplify  data 

•  Provide  access  to  underlying  data  by  selecting  icon 

•  Enhance  cognition  process 

•  Need  to  be  tailored  to  specific  need,  for  example; 

Showing  sensitivity  by  using  a  slider  to  set  decision  values,  with  color  of  slider 
varying  depending  on  sensitivity  of  plan  to  variable. 

Weather  patterns  including  standard  meteorological  symbols 

Communication  links  and  status  (using  color,  varying  line  widths,  etc.) 

Organizational  charts  depicting  chain  of  command;  resource  responsibility 

Representing  uncertainty 

•  Data  Association 

Assign  data  to  features  of  display.  Modify  display  as  a  function  of  data. 

Provide  ability  to  correlate  data  interactively 

•  Overlays/Grouping 

Mechanism  for  logically  grouping  data 
Used  to  interactively  display  data  of  interest 
Simplify  displays  by  removing  unnecessary  detail 

•  Spreadsheets  &  Forms 

Simple  method  of  presenting  and  entering  numerical  data 

Spreadsheets  should  be  capable  of  representing  graphical  images 

Forms  may  be  graphical  in  nature;  i.e.,  data  can  be  entered  through  selection  of 
graphical  images 

•  Sound 

Supplemental  data  manipulation  channel.  Speech  input/output  a  possibility. 

Provide  focus  as  important  information  becomes  available. 

Convey  qualitative  information  using  attributes  such  as  tone,  volume,  and  repeat 
rate. 
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Animation 


Provide  an  understanding  of  time-dependent  data 

Useful  for  providing  overview  of  a  plan,  including  contingencies 

Display  potential  enemy  responses  to  plan,  with  possible  counter  measures 

•  3D  Graphics 

Needed  when  altitude  effects  important 

Provide  added  dimension  for  displaying  multi-variate  data  or  pictorials 
Difficult  to  manipulate  3D  objects  on  2D  displays 

•  Storage/Data  Access 

Need  to  efficiently  and  reliably  access  data 

Use  visual  sign-posts  such  as  icons  to  provide  access  to  data 

Need  to  interactively  associate  data  with  displays 

Need  to  store  intermediate  plans 

Obtain  pertinent  tactical  strategy 

Obtain  past  plans  and  results,  both  friendly  and  enemy 
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3  Maps 


3.1  Introduction 

The  previous  section  examined  the  planning  process  from  a  high  level.  In  this  and  the 
following  sections,  we  begin  to  look  at  the  details  of  the  planning  process.  One  of  the 
fundamental  requirements  is  the  ability  to  display  maps. 

Almost  all  planning,  decision  making,  and  situation  assessment  tasks  in  the  Air  Force 
involve: 

•  A  knowledge  of  the  location  of  objects,  and 

•  Relating  objects  to  their  spatial  environment. 

One  can  easily  see  the  central  importance  of  maps  to  almost  every  task  in  the  Air  Force.  The 
primary  features  displayed  on  most  maps  consist  of: 

c  Elevation  of  the  underlying  terrain, 

•  Nature  of  ground  cover  -  desert,  shrubs,  forest  etc., 

•  Position  and  extent  of  natural  features  -  lakes,  riv^rs,  oceans  etc., 

•  Position  and  extent  of  man-made  features  -  roads,  railway  tracks,  bridges,  runways, 
towns  etc.,  and 

«  Location  and  nature  of  strategic  /  tactical  resources  and  threats  -  airbases,  radar,  SAM 
sites  etc. 

3.2  Display  Criteria 

The  use  of  advanced  color  grap-’iics  techniques  is  essential  for  presenting  this  diverse  range 
of  information.  First,  we  must  set  some  general  guidv;a.„-5  for  the  display  of  maps,  so  that  the 
presented  infonhation  can  be  easily  assimilated  by  the  user. 

1.  Visual  Codes  -  We  should  make  the  best  use  of  visual  cues  such  as  color,  texture  and 
icons,  so  that  the  user  needs  minima!  conscious  thought  for  locating  and  assimilating 
geographic  information. 

2.  Information  Masking  -  We  must  avoid  the  risk  of  visual  and  information  overload  of 
the  user,  by  selectively  presenting  only  infoimation  which  is  relevant  to  the  current  user 
task. 
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3.  Zoom  and  Pan  -  The  user  should  be  able  to  rapidly  obtain  a  display  of  the  required 
region  of  the  map,  at  the  desired  level  of  detail.  This  task  should  require  as  little 
interaction  with  the  user  as  possible.  Access  to  the  stored  map  data  should  also  be 
accelerated  by  the  use  of  appropriate  algorithmic  and  data  structuring  techniques. 

4.  Derived  Information  -  V/e  must  provid'  means  for  the  user  to  query  the  map 

application  for  any  information  which  er».a  •  .-.’.sonably  deduced  from  the  underlying 
map  information.  Examples  include  distance,  geographically  weighted 

measures  such  as  time  of  flight  etc. 

5.  Display  Customization  -  The  user  should  be  able  to  dynamically  change  the  visual 
representation  of  the  mapped  information,  '  b  .  minimal  effort.  Examples  include  color 
and  texture  coding,  icon  classification  ciz.  “lignificant  advantage  car.  be  gained  by 
being  able  to  dynamically  link  secondary  viruai  properties  of  the  displayed  icons  (such 
as  size,  orientation,  blink  rate  etc.)  to  user  selectable  attributes  of  the  underlying  object 
(such  as  altitude,  velocity,  remaining  fuel,  weapons  cargo  etc.).  For  example,  on  the 
outward  leg  of  a  bombing  run,  the  size  of  the  displayed  aircraft  icon  could  represent 
the  quantity  of  munitions,  whereas  on  the  return  it  could  denote  the  quantity  of 
remaining  fuel. 

6.  Dynamic  Status  Display  ~  We  must  provide  the  means  to  represent  dynamically 
changing  attributes  of  the  displayed  objecb?.  We  must  also  adapt  suitable  display 
techniques  so  that  the  user  can  easily  perceive  the  nature  and  magnitude  of  changes  in 
those  atu  i butes.  Some  of  these  attributes  could  be  tracked  by  information  received  from 
outside  the  system,  and  yet  i;r«iers  could  be  updated  from  a  script  which  may  for  example 
replay  a  planned  mission. 

3.3  Displaying  Topography 

Topography  is  a  description  of  the  elevation  at  each  position  of  the  earth’s  surface.  A 
knowledge  of  the  topography  along  mission  routes  and  surrounding  locations  of  possible  enemy 
encounters  is  crucial  for  planning  successful  missions,  as  well  as  for  selecting  basic  strategies  for 
attack  and  defense.  For  example,  narrow  valleys  provide  excellent  opportunities  for  evading 
radar,  but  also  create  a  hazard  for  flying.  Also,  being  on  the  leeward  side  of  a  mountain  with 
respect  to  the  location  of  a  radar  or  SAM  site  is  clearly  advantageous.  While  flying  Close  Air 
Support  missions,  a  knowledge  of  the  slope  and  texture  of  the  underlying  terrain  is  important, 
since  it  affects  the  nature  of  the  land  battle,  and  hence  impacts  the  type  of  required  support. 

Topography  can  be  displayed  in  a  two  dimensional  or  a  three  dimensional  form.  A  2D 
display  is  obtained  by  dividing  up  the  land  /  water  surface  into  regions  which  lie  within  selected 
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bands  of  elevations.  It  should  be  pc^ssible  for  the  user  to  select  the  number  of  displayed  ranges, 
as  well  as  the  bounds  for  each  lange  of  elevation.  This  is  not  possible  in  most  current  systems, 
since  the  display  form  of  the  map  is  usually  pre-computed  using  pre-selected  and  uniformly 
spaced  elevation  ranges.  A  two  diihensional  display  of  the  elevation  ranges  is  possible  in  the 
following  ways: 

1.  Filled  Map  -  by  color  coding  each  range  distinctly,  or  as  a 

2.  Contour  Map  -  by  drawing  the  contours  which  demarcate  esch  range,  and  labelling  each 
contour  with  its  elevation. 

A  three  dimensional  topographic  display  is  possible  in  one  of  the  three  following  ways: 

1.  Wireframe  model  -  a  uniformly  spaced  wire  grid  is  assumed  to  be  moulded  to  the  three 
dimensional  terrain  surface,  and  the  resulting  curved  lines  constitUi.ng  the  deformed 
grid  are  projected  onto  the  display  plane. 

2.  Illuminated  Polyhedral  model  -  the  three  dimensional  terrain  is  approximated  by  3D 
polyhedra,  which  arc  then  illuminated  suitably,  and  the  resulting  shaded  polygonal  faces 
of  the  polyhedra  are  projected  onto  the  display  plane, 

3.  Texture  Mapped  model  -  photographs  of  several  perspective  views  of  the  actual  terrain 
are  image  compressed,  and  mapped  onto  the  polygonal  faces  of  the  modelled  terrain. 
This  results  in  a  more  realistic  display  of  the  colors  and  texture  of  the  target  terrain. 
With  texture  mapping,  it  is  also  possible  to  account  for  seasonal  variations,  as  well  as 
variations  due  to  time  of  day  or  \  .ather  effects  on  illumination  and  visibility,  in  order 
to  generate  a  very  realistic  display.  Such  techniques  permit  the  user  to  assimilate  these 
related  items  of  information  effortlessly,  and  thus  account  for  these  factors  duntig 
planning  and  situation  assessment. 

While  displaying  either  a  filled  map  or  a  contour  map,  one  has  to  be  careful  not  to  obscure 
the  display  of  other  desired  features  such  as  roads,  lakes,  political  boundaries  etc.  It  is  important 
to  permit  the  user  to  customize  the  display,  so  that  he  can  highlight  the  desired  information.  At 
the  same  tinic,  it  is  important  to  adhere  to  some  relatively  uniform  standards  for  display,  so  that 
different  personnel  can  interact  during  decision  making,  and  can  simultaneously  refer  to 
common  points  of  interest  on  the  display.  The  use  of  customizable  and  default  display  profiles, 
as  well  as  convenient  temporary  highlight  modes,  can  significantly  enhance  the  planning  and 
decision  making  environment. 

3.4  Displaying  Derrain  and  Cultural  Features 

Features  of  the  terrain  which  are  of  interest  include  both  natural  and  man-made  objects. 
Natural  area  features  such  as  soil  type  and  ground  cover  can  be  depicted:  (i)  by  color  coding  the 
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area,  (ii)  by  filling  the  area  with  a  selected  textural  pattern,  or  (iii)  by  randomly  distributing  icons 
of  the  selected  type  across  the  area.  ITie  last  method  can  also  be  used  to  depict  the  density  of  the 
displayed  object  in  that  area  by  a  corresponding  density  of  its  icons  in  the  display.  Other  area 
features  correspond  to  bodies  of  water  such  as  lakes,  rivers  and  oceans.  Such  areas  are  almost 
always  color  coded,  with  the  color  also  depicting  depths  within  different  regions  in  the  case  of 
lakes  and  oceans.  In  the  case  of  rivers,  we  also  need  to  depict  its  width.  Since  its  width  is  typically 
very  small  compared  to  the  scale  of  the  map,  the  width  needs  to  be  coded  by  some  symbolic 
means  such  as  line  type  or  line  width. 

Man  made  objects  are  often  the  most  interesting  features  on  a  map,  and  include  such  objects 
as  roads,  railroads,  bridges,  airfields,  towns,  factories,  depots,  SAM  sites,  radar  etc.  There  are 
well  accepted  icons  for  depicting  such  objects.  It  is  useful  to  provide  means  for  the  user  to  add 
icons  to  the  display  library,  as  well  as  enable  him  to  modify  the  icons  associated  with  a  selected 
category  of  objects.  A  significant  benefit  can  be  obtained  by  permitting  the  user  to  dynamically 
link  secondary  visual  properties  of  the  displayed  icons  (such  as  size,  orientation,  color,  blink  rate 
etc.)  to  desired  attributes  of  the  underlying  obje:’t  (such  as  altitude,  velocity,  remaining  fuel, 
weapons  complement,  population,  traffic  ^atcs,  load  bearing  capacity  etc.). 

3.5  Coordinate  Systems  and  Projections 

The  representation  and  display  of  geographic  information  is  often  based  on  a  flat  earth  or 
rectangular  (LAT  LONG)  coordinate  system,  when  the  user  is  v^■c^V'ng  relatively  small  areas. 
This  permits  rapid  assimilation  of  the  relative  distances  and  orientations  between  points  of 
interest.  However,  cis  the  viewing  area  increases,  or  as  we  approach  regions  farther  from  the 
Equator,  the  distortions  become  unacceptably  large  if  we  ignore  the  curvature  of  the  earth’s 
surface. 

Using  current  technology,  it  is  possible  to  render  a  map  on  a  spherical  surface,  which  is  then 
displayed  on  a  flat  screen  as  it  v/ould  appear  to  a  camera  at  the  location  of  the  user,  and  with  his 
perspective.  The  underlying  map  information  would  naturally  be  stored  in  a  geocentric 
coordinate  system.  However,  over  the  years  many  standard  projections  have  been  devised  by 
cartographers  in  order  to  render  maps  on  paper.  The  following  projections  are  some  of  the  more 
familiar  ones; 

1 .  Mercator  -  the  globe  is  conformally  mapped  onto  a  cylinder.  Land  shapes  are  displayed 
correctly,  but  the  areas  of  landmasses  are  greatly  exaggerated  towards  the  poles. 

2.  Lssnbert  Equal  Area  -  the  globe  is  projected  onto  a  plane  tangential  to  it.  It  shows  true 
distances  close  to  the  point  of  tangency,  but  compresses  them  greatly  as  we  move  farther 
away  from  the  center. 
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3.  Goode  -  it  is  an  equal  area  projection.  The  parallels  of  latitude  are  all  depicted  with  the 
same  scale,  with  breaks  in  them  to  separate  the  major  land  masses.  Each  land  mass  is 
depicted  about  a  separate  central  meridian,  which  permits  minimal  distortion  for  each. 


4.  Robinson  -  this  recently  devised  projection  attempts  to  simultaneously  minimize 
distortions  in  shape,  relative  sizes,  orientations,  and  relative  distances. 

3.6  Displaying  and  Scanning  Large  Areas 

A  theater  of  air  operations  typically  spans  an  extremely  large  area,  and  viewing  a  map  of  the 
entire  theater  with  complete  detail  would  often  be  neither  practical  nor  even  useful.  Hence,  the 
user  needs  to  be  able  to  selectively  display  just  the  region  of  interest.  If  the  selected  region  is 
large,  then  the  user  needs  to  be  able  to  disable  the  display  of  selected  categories  of  objects.  Also, 
as  the  focus  of  attention  moves,  or  changes  in  scope,  it  becomes  necessary  to  zoom  and  /  or  pan 
across  the  map. 

The  speed  and  convenience  of  effecting  these  display  changes  are  influenced  by  two  factors: 

1.  User  Interaction  -  The  fastest  scheme  that  is  possible  with  available  technology  is  for 
the  user  to  be  able  to  zoom  out  /  in  with  fixed  magnification  /  demagnification  factors, 
using  a  pop-up  menu  attached  to  a  mouse  button.  The  user  could  also  lasso  an  arbitrary 
'rectangular  region  of  the  displayed  map  using  a  mouse,  and  have  that  portion  fill  the 
entire  field  of  view.  Additionally,  the  user  could  selectively  enable  /  disable  desired 
categories  of  map  objects,  by  using  a  programmable  pop-up  menu  attached  to  a  mouse 
button,  or  by  selecting  or  deselecting  specific  objects.  This  could  be  aided  by  associating 
attributes  and  classes  with  each  instance  of  a  displayable  object.  Additional  speedup 
can  be  achieved  by  permitting  the  user  to  set  named  tags  with  map  locations,  and 
enabling  the  user  to  directly  change  the  focus  of  the  displayed  map  to  a  tagged  location, 
or  to  a  specified  latitude  and  longitude.  During  operations  in  which  the  user  wishes  to 
pan  across  large  areas  of  the  map,  we  can  obtain  a  significant  speedup  in  displaying 
the  map  by  suppressing  the  display  of  many  unnecessary  details  and  features. 
Unfortunately,  very  few  of  these  desirable  features  are  supported  by  existing  map 
display  applications. 

2.  Storage  Retrieval  -  Map  databases  are  inherently  very  large,  and  the  time  required  for 
retrieving  desired  portions  of  the  map  for  display  can  be  speeded  up  significantly  by 
using  innovative  data  structuring  techniques,  as  well  as  by  using  innovative  algorithms 
for  locating  desired  fragments  of  the  map  within  the  storage  medium.  This  would 
provide  speedups  above  and  beyond  that  provided  by  faster  hardware  alone,  and  is  also 
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necessary  since  a  user’s  expectations  about  the  level  of  detail  in  the  map  go  up  as  soon 
as  hardware  becomes  faster. 

3.7  Level  of  Detail 

As  discussed  earlier,  it  is  not  desirable  to  display  all  the  available  information  on  a  map,  in 
order  to  avoid  an  information  overload  of  the  user.  Some  of  the  detail  can  be  suppressed  by  the 
user,  by  selectively  disabling  individual  items  or  entire  categories  of  objects.  Additionally,  there 
are  two  automatic  methods  which  cari  be  used  for  suppressing  detail. 

1.  Boundary  Detail  Scaling  -  It  is  possible  to  display  linear  features,  and  the  boundaries 
of  area  features,  with  a  number  of  segments  which  is  appropriate  to  the  scale  of  the 
display.  It  is  possible  to  approximate  area  features  with  a  series  of  bounding  polygons, 
progressively  having  a  greater  number  of  segments,  and  approximating  the  feature  with 
a  smaller  maximum  deviation.  When  the  scale  of  the  display  is  very  small,  it  may  be 
necessary  to  generate  additional  jagged  edges  using  fractal  interpolation,  for  displaying 
realistic  boundaries  for  area  features  such  as  lakes  etc. 

2.  Context  Sensitive  Filters  -  We  can  expect  that  the  map  display  application  is  invoked 
in  conjunction  with  a  user  task  menu  system,  which  enables  the  user  to  undertake 
predefined  categories  of  command,  planning,  or  decision  type  tasks.  It  would  be  fairly 
straightforward  to  custom  design  display  profiles  for  each  task  category,  which  would 
specify  what  should  be  displayed  for  such  a  task.  It  is  also  important  to  provide 
mechanisms  for  end  users  to  easily  modify  the  task  categories,  as  well  as  the  display 
profile  for  each  category. 

3.8  Secondary  and  Inferred  Attributes 

Using  object  oriented  techniques,  it  is  possible  to  generate  smart  maps  which  can  be  queried 
for  information  related  to  the  underlying  map,  as  the  following  examples  illustrate. 

1.  The  user  could  select  two  locations  on  the  map,  and  inquire  as  to  whether  they  are 
connected  by  road  /  rail,  or  what  the  estimated  travel  time  or  fuel  requirement  between 
those  destinations  is  by  road  /  rail  /  air.  The  answers  to  such  questions  could  take  into 
account  current  status  of  road  conditions,  weather,  known  enemy  activity,  type  of 
transport  vehicle,  mission  directives,  policy  etc. 

2.  The  user  could  enter  the  details  of  a  mission  plan,  such  as  flying  routes,  waypoints,  start 
and  synchronization  times,  and  other  supporting  troop  activity.  Then  he  could  use  the 
map  and  mission  plan  information  as  a  base  to  animate  the  probable  sequence  of  events. 
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as  well  as  to  display  dynamic  status  information  such  as  remaining  fuel,  cumulative 
survival  probability,  instantaneous  threat  level,  weapons  complement  etc. 

3.9  Data  Accuracy  and  Automated  Map  Generation 

The  primary  sources  of  map  information  are:  (i)  Distal  Terrain  Elevation  Data  (DTED), 
and  (ii)  Digital  Feature  Analysis  Data  (DFAD),  both  provided  by  the  Defense  Mapping  Agency 
(DMA),  as  well  as  data  provided  by  the  US  Geological  Surv^  (USGS).  These  data  often  contain 
i  naccuracies  due  either  to  the  poorer  resolution  obtainable  in  older  surveys,  or  as  a  result  of  man 
made  or  natural  changes  to  the  area.  Sometimes,  these  inaccuracies  can  result  in  annoying 
errors  in  the  displayed  map,  such  zs  lakes  without  closed  boundaries,  or  bridges  not  aligned  with 
the  roads  th^  are  on.  There  are  two  major  remedies  to  these  errors,  and  both  involve 
automating  the  map  making  process  to  a  greater  degree  than  at  present.  The  Crst  is  to  develop 
new  image  processing  algorithms  which  will  enable  much  of  the  map  data  to  be  corrected  by 
reconciliation  with  high  resolution  satellite  images  of  the  terrain  obtained  using  different  bands 
such  as  visible  light,  ultraviolet,  and  in&ared.  The  second  method  involves  incorporating 
semantic  constraints  with  various  object  categories,  such  as:  (i)  bodies  of  water  must  have  a 
closed  boundary,  and  (ii)  a  bridge  must  align  with  an  associated  road  or  railway  track  etc. 
Algorithms  can  be  devised  to  check  these  constraints,  as  well  as  to  correct  any  detected 
violations  of  these  rules. 

3.10  Storage  Representation 

The  internal  form  in  which  the  map  is  stored  has  a  significant  impact  on:  (i)  the  speed  with 
which  the  map  can  be  accessed  and  displayed,  (ii)  the  hardware  requirements  (size  and  speed) 
for  storage,  (iii)  the  flexibility  with  which  the  map  data  can  be  modified,  and  (iv)  the  ease  with 
which  other  applications  can  make  use  of  the  map  data.  In  Section  3.8  we  showed  examples  of 
applications  which  rely  on  querying  and  /  or  updating  the  map  database. 

The  simplest  form  in  which  map  data  can  be  stored  is  as  a  raster  of  points  obtained  by 
digitizing  desired  maps  at  fixed  scales.  This  is  the  least  useful  representation:  since  it  uses  most 
storage,  produces  unsatisfactory  results  for  changing  scales  (with  large  computational  cost),  and 
cannot  be  used  as  a  basis  for  computing  secondary  inferred  attributes. 

The  most  preferable  scheme  would  have  the  following  features. 

•  Displayable  features  on  the  map  should  be  stored  in  an  object-oriented  form,  with  size 
and  placement  specified  in  a  manner  which  is  independent  of  the  viewing  scale  or 
resolution. 
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•  The  perimeter  of  area  features,  as  well  as  the  skeleton  of  h'near  features  should  be 
specified  as  a  hierarchy  of  successively  closer  polygonal  (segmented)  approximations, 
and  independently  of  the  resolution  of  the  display  device. 

•  Displayable  objects  should  have  tags  associated  with  them,  which  specify  the  type  of 
each  object.  These  tags  could  be  used  to  selectivefy  enable  the  display  of  desired  classes 
of  objects,  as  well  as  for  customizing  the  color,  linewidth,  fine  style,  texture  and  other 
display  attributes. 

•  Displayable  objects  can  have  other  items  of  information  associated  with  them,  as  well 
as  links  to  related  objects:  which  can  be  used  for  computing  secondary  inferred 
attributes. 

•  The  data  structures  for  storing  the  map  should  be  organized  to  permit  very  rapid  access 
to  desired  coordinates,  for  panning  across  areas  of  the  map,  and  also  for  rapidly 
changing  scales.  Some  techniques  which  are  known  for  this  task  include  Indexed  Search 
Trees  and  Hashing. 

3.11  Three  Dimensional  Information 

Topographic  information  about  a  map  is  most  naturally  available  in  a  three  dimensional 
form.  In  section  3.3  we  discussed  various  techniques  for  displaying  topography.  In  most 
planning  related  tasks,  the  map  is  viewed  at  such  a  scale  that  just  a  symbolic  display  of  the 
altitudes  (using  contours  or  color  coding)  is  adequate.  The  scale  of  percepUon  for  such  tasks 
focuses  on  the  two  dimensional  attributes  almost  exclusively. 

Occasionally,  it  becomes  important  to  visualize  the  effects  of  Terrain  Masking,  when  for 
example  high  mountains  make  aircraft  in  some  area  invisible  to  radar.  Also,  when  planning  a 
flight  route  through  a  narrow  valley,  it  beco.mes  important  to  visualize  the  detailed  three 
dimensional  features  of  the  terrain,  in  order  to  determine  the  feasibility  of  certain  flight 
patterns,  or  of  ffying  certain  types  of  aircraft.  Sometimes,  weather  related  phenomena  such  as 
storms  or  wind  shear  are  also  greatly  influenced  by  the  topography  of  the  terrain,  whence  it  can 
be  important  to  view  the  terrain  in  3D  for  such  applications  as  well. 

3.12  Summary 

The  following  is  a  brief  summary  of  the  graphics  requirements  for  maps. 

•  2D/3D  Pictorials 

*  A  2D  display  of  topography,  showing  elevation  ranges,  can  be  achieved  by: 
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color  coding  each  range  of  elevations 

using  contours  to  demarcate  each  range  and  labelling  each  contour  with  its  elevation 

•  A  3D  terrain  display  can  be  important  for  such  applications  as  terrain  masking, 
mission  rehearsal,  and  weather  patterns.  A  3D  display  of  topography  can  be 
achieved  by: 

projecting  a  wireframe  model  of  the  3D  terrain  surface  on  to  the  display  plane 
projecting  a  3D  shaded  polygon  model  of  the  terrain  surface  on  to  the  display  plane 

photographs  of  the  actual  terrain  or  terrain  features  (roads,  buildings)  are  “texture 
mapped”  on  to  the  polygons  of  the  modelled  terrain 

•  Display  Customization 

•  Change  visual  information  associated  with  map  quickly  (icon  types,  icon 
classifications,  color,  line  widths) 

•  Coordinate  Systems 

•  Rapid  and  accurate  conversions  of  different  coordinate  representations 

•  User  interactions 

•  Zoom  out/in  with  fixed  magnification/demagnification  factors  (popup  menu,  lasso) 

•  Selectively  enable/disable  individual  objects  or  categories  of  objec  s  (popup  menu, 
pointing  to  objects) 

•  Select  a  specific  latitude  and  longitude 

•  Associate  name  tags  to  map  locations  and  select  name  tag  to  “focus”  display 

•  Queiy  map  for  data(attributes)  related  to  objects  on  map. 

•  Display  elevation  as  you  move  along  terrain  surface 

•  Storage/Data  Access 

•  Innovative  data  structures  and/or  hardware  for  compressing  map-related  data  and 
rapidly  accessing  portions  of  a  map 
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Cost-€ffective  data  storage 


•  Level  of  Detail 

•  Ability  to  display  large  area  maps  and  then  rapidly  display  smaller  area  maps,  with 
display  automatically  accounting  for  difference  in  scale 

•  Selectively  disable/enable  individual  objects  or  categories  of  objects  (overlays) 

•  Display  linear  features  and  boundaries  of  area  features,  with  the  number  of 
segments  appropriate  for  the  display  scale 

•  Modelling 

•  Automation  techniques  for  map  generation  process  and  map  accuracy 

•  Image  processing  algorithms  for  photographic  data 

•  Heuristic  rules/constraints  for  categories  of  objects,  e.g.,  bridge  must  align  with 
road  or  railroad  track 

•  Geometric  Representation 

•  Displayable  features  stored  in  object-oriented  form 

•  Objects  have  wide  variety  of  attributes  associated  with  them(color,  linewidth,  line 
style)  which  can  be  quickly  accessed,  displayed,  or  modified 

•  Efficient  geometric  data  structures  for  changing  scales  rapidly,  panning  across 
areas,  and  accessing  desired  coordinates. 

•  Data  Association 

•  Objects  can  have  links  to  other  objects.  For  example,  link  the  visual  property  of  an 
icon(size,  blink  rate)  with  a  selectable  attribute  of  an  object(remaiiiing  fuel,  amount 
of  weapons). 

•  Special  Algorithms 

•  Calculate  information  which  can  be  deduced  from  underlying  map  data,  such  as 
distances  and  connectivity. 
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4  Threat  Zones 


4.1  Introduction 

Al!  enemy  threats,  from  the  largest  to  the  smallest,  can  potentially  have  a  negative  impact  on 
a  mission.  The  volume  in  which  a  threat  is  effective  is  termed  its  threat  zone.  The  relevant 
properties  of  a  threat  depend  on  several  factors,  including 

•  its  firepower, 

•  its  sensing  capabilities, 

•  its  mobility, 

•  the  impact  of  geographic  and  meteorologic  factors  on  the  above  items, 

•  potential  for  cooperation  between  multiple  threats,  and 

•  the  nature  of  the  forces  it  threatens. 

Accurate  display  and  interpretation  of  threat  zone  properties  is  critical  to  ensure  effective 
mission  planning  and  execution.  What  to  display  is  a  compromise  between  the  magnitude  of  the 
threat  to  the  mission,  how  accurately  the  threat’s  properties  are  known,  how  threat  display 
enhances  (or  detracts  from)  other  objects  being  displayed  (e.g.,  weather,  geography,  front  line  of 
troops,  low-level  transit  routes,  and  targets),  and  the  overall  complexity  of  the  display.  This 
section  describes  several  techniques  for  displaying  threats  effectively. 

4.2  Current  Techniques 

Most  threat  displays  use  a  constant  intensity  image  to  represent  a  threat,  displaying  just  the 
threat  boundary  (outline  mode)  or  a  series  of  spokes  radiating  from  the  threat  itself.  These 
display  techniques  suffer  from  several  problems: 

•  They  do  not  account  for  different  threat  magnitudes  in  different  parts  of  the  overall 
threat  zone.  For  example,  we  have  not  seen  any  displays  of  radars  that  visually  account 
for  the  1/^  nature  of  the  radar  signal. 

•  They  do  not  account  for  threats  of  different  strengths.  A  SAM  site  has  a  larger  threat 
zone  than  a  soldier  with  a  rifle,  but  if  both  are  displayed,  they  will  be  displayed  with 
much  the  same  properties, 

•  They  do  not  consider  constructive  or  destructive  interactions  between  overlapping 
threats. 
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•  There  is  no  ability  to  synthesize  higher  level  images  from  the  sum  of  the  individual 
threats. 

•  No  systems  we  have  seen  allow  the  user  to  reduce  the  visibility  of  threats,  other  than 
to  totally  eliminate  them  from  the  display. 

•  No  systems  we  have  seen  can  display  threat  zones  in  three  dimensions. 

•  No  systems  we  have  seen  account  for  threat  movement. 

•  No  systems  we  have  seen  account  for  uncertainty  in  the  information  about  threat 
properties. 

•  Few  display  distinctions  are  made  between  different  types  of  threats,  say  a  SAM  site 
vs.  a  radar  site. 

•  They  do  not  display  ownship  vulnerability  to  threats. 

•  No  systems  we  have  seen  (or  heard)  use  sound  to  represent  threat  information. 

4.3  Threat  Magnitudes 

A  threat  does  not  have  the  same  effectiveness  over  its  entire  range.  For  instance,  radar  power 
is  proportional  to  the  inverse  of  the  square  of  the  distance  from  the  source.  Guns  and  missiles 
are  not  as  accurate  near  the  limits  of  their  range.  Also,  different  threats  have  different  maximum 
potential  for  lethality.  Threat  display  should  reflect  these  differences  and  gradients. 

4.4  Higher  Level  Display  of  Threat  Information 

Threat  zones  often  overlap,  and  the  danger  within  that  overlapping  area  is  probably  not  the 
danger  associated  with  just  one  of  the  overlapping  threats.  Tvo  or  more  overlapping  missile 
threat  zones  might  increase  the  danger,  since  more  missiles  might  be  fired  at  your  planes.  On  the 
other  hand,  overlapping  radar  threats  might  interfere  with  each  other  reducing  their 
effectiveness  in  the  area  of  overlap. 

The  potential  for  constructive  or  destructive  interaction  implies  that  threats  should  not  be 
considered  in  isolation,  but  should  be  treated  as  a  single,  larger  entity.  This  is  especially 
important  during  higher  level  planning,  where  the  details  of  individual  threats  may  be  less 
important  than  the  overall  danger  of  a  situation.  Several  combination  rules  for  multiple 
overlapping  threats  exist.  Three  simple  ones  are  2fr, ,  max  ,  and  ,  where  T,  is  the 

/th  displayed  threat. 
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4.5  Threat  Visibility 

In  extremely  congested  theaters,  such  as  Europe,  the  number  of  threats  may  be  so  great  that 
display  of  all  threats,  or  even  a  substantial  fraction  of  the  total  threats  in  an  area,  clutters  the 
display,  and  may  cause  the  planner  to  lose  focus  of  other  important  features.  Some  way  to 
selectively  subdue  threat  display  is  necessary.  Some  techniques  to  consider  are 

•  displaying  only  those  threats  to  which  mission  resources  are  vulnerable, 

•  partitioning  threats  into  types  (e.g.,  SAMs  vs,  radars  vs.  anti-aircraft), 

•  accounting  for  the  three-dimensional  nature  of  the  threat  envelope,  altitude  of  friendly 
resources  and  interaction  with  geography  (terrain  masking). 

•  suppressing  display  of  threats  that  do  not  overlap  planned  routes, 

•  reducing  the  displayed  intensity  of  selected  threats,  and 

•  displaying  a  hierarchy  of  threats,  and  only  displaying  those  matching  some 
user-specified  display  criteria. 

Techniques  that  cycle  between  different  overlays  may  be  beneficial  as  well.  For  instance, 
enemy  threats  may  be  displayed  in  one  overlay,  weather  in  another,  and  geography  in  yet 
another.  The  three  overlays  might  be  cycled  every  second  or  two  to  present  the  planner  with  all 
the  information,  but  prevent  the  display  from  being  too  cluttered.  The  planner  should  be  able  to 
select  how  many  overlays  are  displayed,  what  their  contents  are,  in  what  order  they  will  be 
displayed,  whether  they  will  be  statically  displayed,  and  if  dynamic,  what  their  cycle  frequency 
will  be. 

4.6  Three-dimensional  Threat  Display 

Threat  zones  are  typically  displayed  in  just  two  dimensions,  yet  they  have  properties  that 
make  it  important  to  view  them  in  three  dimensions,  especially  during  lower  levels  (e.g., 
unit-level)  of  the  planning  process.  Elevation  effects,  such  as  terrain  masking,  cannot  be 
understood  properly  in  just  two  dimensions.  The  threat  zone  may  not  be  hemispherical  (even 
neglecting  terrain  and  weather  effects),  especially  for  aerial  threats.  Also,  weather  and  terrain 
effects  typically  manifest  themselves  at  different  altitudes,  making  a  two-dimensional  display 
an  over-simplification.  Viewing  threats  in  three  dimensions  from  different  perspectives  can  be 
of  high  value  during  mission  rehearsal. 

4.7  Representing  Uncertainty  in  Threat  Properties 

Information  regarding  enemy  threats  is  not  absolute.  Threats  move,  their  firepower 
increases  and  decreases,  and  sometimes  intelligence  does  not  have  enough  information  to  say 
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for  certain  just  what  the  properties  of  the  threat  are.  These  factors  contribute  to  uncertainty 
about  threat  capabilities.  Planners  should  have  some  way  of  determining  the  uncertainty  of 
various  threat  capabilities. 

Bonissone  and  Dutta  [7]  view  uncertain  information  in  their  reasoning  system  by  plotting 
both  the  evidence  supporting  and  refuting  a  conjecture  as  a  small  bar  chart.  The  supporting 
value  is  plotted  from  the  left  edge  of  the  interval,  while  the  refuting  value  is  plotted  from  the  right 
edge,  both  in  white.  The  region  between  the  bars  represents  the  amount  of  uncertainty  and 
colored  black^  Overlap  of  the  supporting  and  refuting  values  represents  an  error  in  the  rule  base 
of  the  reasoning  system.  The  viewer  can  tell  quickly  whether  the  evidence  available  generally 
supports  or  refutes  the  conjecture  it  represents,  and  how  uncertain  that  evidence  is  by  the 
position  and  size  of  the  black  bar.  The  larger  the  black  bar,  the  more  uncertain  the  information 
is.  The  more  to  the  right  it  is,  the  more  the  evidence  is  positive,  to  the  left,  negative.  In  LOTTA 
[5],  threat  intent  is  represented  by  five  different  variables,  engage-rtow,  engage-later,  influence, 
evade,  and  non-reactive.  For  each  enemy  airplane,  all  five  variables  are  plotted  as  described 
above  in  a  small  window. 

4.8  Distinguishing  Threat  lypes  Using  Animation 

Different  types  of  threats  have  different  properties.  The  planner  should  be  able  to  display 
those  differences  if  he  chooses.  Radar  sites  might  be  displayed  with  a  sweeping  line  from  the 
center  to  the  edge  of  the  threat,  similar  to  the  radar  displays  popular  in  Hollywood  movies. 
Anti-aircraft  guns  might  be  displayed  with  random  flashing  dots  throughout  the  displayed 
threat  zone.  SAM  sites  might  be  displayed  with  dots  that  trace  a  line  from  the  center  to  the  edge 
of  the  threat  zone,  with  the  dot  fading  out  as  the  edge  of  the  threat  zone  is  reached. 

4.9  Displaying  Resource  Vulnerability 

If  multiple  types  of  aircraft  are  being  deployed  on  a  mission,  they  are  likely  to  have  different 
vulnerability  to  enemy  threats.  It  is  important  to  understand  the  relationship  between  the 
different  types  of  threats  and  the  different  types  of  aircraft.  Some  possibilities  for  displaying 
these  relationships  are 

•  Color  each  aircraft’s  route  to  indicate  its  vulnerability  to  threats  at  each  point  along  the 

route.  Threat  display  becomes  potentially  easier,  since  there  is  no  strict  requirement  to 

represent  the  lethality  of  each  individual  threat.  Aircraft  display  for  different  types  of 

aircraft  can  be  shown  simultaneously  in  multiple  windows.  This  has  the  potential 

1.  Overlap  of  the  supporting  and  refuting  values  represents  an  error  in  the  rule  base  of  the  reason¬ 
ing  tystem. 
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disadvantage,  however,  of  spatially  separating  different  pieces  of  information  that 
should  be  registered  to  the  map  section  being  viewed  and  to  each  other. 

•  Select  the  display  of  different  types  of  aircraft,  and  display  the  threats  based  upwn  the 
vulnerability  of  the  aircraft  being  displayed.  This  has  the  advantage  of  leaving  the  rc  Ite 
display  for  other  factors,  such  as  fuel  load,  although  it  sacrifices  the  ability  to  view  all 
aircraft  and  their  vulnerability  to  threats  at  the  same  time. 

4.10  Sound 

The  purpose  of  sophisticated  threat  display  is  to  transfer  a  large  amount  of  information  to 
the  planner  in  a  form  he  or  she  can  utilize.  Very  few  systems  (even  outside  the  planning  realm) 
make  use  of  sound  to  represent  information,  however.  Humans  are  good  at  recognizing  and 
acting  upon  two  inputs  from  different  senses  (e.g.,  visual  and  sound).  They  are  less  good  at 
recognizing  and  acting  upon  two  inputs  from  the  same  sense  (e.g.,  color  and  animation).  It 
makes  sense  to  take  advantage  of  humans’  natural  signal  processing  capabilities  where  possible. 

Bly  [4]  has  shown  that  people  can  cluster  multivariate  data  accurately  based  upon  several 
sound  properties,  such  as  amplitude  and  frequency.  Audible  information  can  be  quite  useful  in 
situations  where  the  user  can’t  shift  his  or  her  visual  focus  from  specific  displayed  information 
(either  the  primary  window  or  a  specific  object). 

Listed  below  are  some  possible  uses  for  sound  in  threat  display  activities. 

•  If  a  planner  wants  detailed  information  about  a  threat,  rather  than  selecting  it  and 
having  to  shift  focus  to  a  secondary  window,  much  of  the  data  can  be  presented  aurally, 
using  speech  synthesis  techniques. 

•  Overall  threat  level  during  mission  rehearsal  can  be  presented  aurally,  rather  than 
adding  clutter  to  the  display. 

•  The  user  might  attach  a  spatially  dependent  parameter  to  a  sound  variable.  As  the 
pointer  is  moved  about  the  screen,  the  value  of  that  variable  could  be  heard,  rather  than 
seen. 


4.11  Summary 

In  this  section  we  considered  several  criteria  for  effective  display  of  enemy  threats.  Those 
criteria  can  help  planners  distinguish  threat  Qpes,  suppress  unnecessary  information,  tailor  the 
display  to  their  needs,  and  focus  on  information  in  the  display  that  is  critical  to  effective 
planning  and  execution  of  Air  Force  missions. 
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The  following  is  a  brief  summary  of  the  graphics  requirements  for  representing  threats. 

•  2D/3D  Pictorials 

•  Uncertainty  graphs  to  represent  inaccurate  iiuoraration 

•  Color  routes  according  to  threat  level 

•  Color 

•  Color  contours  to  display  threat  levels  over  area  of  interest 

•  Use  color  gradients  to  represent  overall  threat  level  at  any  given  point, 

•  Adjust  color  saturation  and  intensity  according  to  threat  level 

•  Special  Algorithms 

•  Represent  threat  levels  based  on  interaction  of  threats  with  each  other 

•  Represent  threat  levels  ba.sed  on  interaction  of  threats  with  friendly  forces 

•  Multiple  Displays 

•  Use  to  represent  threat  levels  for  different  aircraft/threat  combinations 

•  Sound 

•  Provides  another  channel  for  information  transfer 

•  Present  level  aurally;  using  spoken  words  or  by  assigning  to  sound  attribute  such 
as  tone  or  volume. 

•  Animation 

•  Radar  site  displayed  as  revolving  line 

•  Anti-aircraft  displayed  as  flashing  dots  in  effective  range 

•  SAM  sites  displayed  with  points  tracing  lines  radially  from  center  of  site.  Lines  fade 
as  threat  level  varies. 

•  3D  Graphics 

•  Use  to  display  threat  envelopes 

•  Use  to  represent  terrain  masking 

•  Overlays/Grouping 
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•  Associate  threats  of  a  particular  type 

•  Associate  threats  of  a  given  lethality 

•  Group  according  to  threat  to  particular  aircraft 
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5  Weather 

5.1  Introduction 

Weather  has  a  major  in.pact  on  visibility,  as  well  as  on  the  maneuverability  of  aircraft. 
Weather  conditions  can  provide  immunity  to  being  seen  by  the  enemy,  winds  can  speed  or  delay 
flying  aircraft,  and  storms  can  present  a  flying  hazard.  Hence,  flight  crew  as  well  as  planners 
need  to  account  for  the  effects  of  weather  while  planning  missions  and  flying  routes. 

5.2  Icons 

There  are  many  icons  which  are  fairly  standard  and  widely  used  for  representing  weather 
related  information.  Examples  include:  (i)  cloud  shapes  for  clouds,  (ii)  slanting  broken  lines  for 
rain,  (iii)  snow  flakes  for  snow,  (iv)  circle  with  outgoing  rays  for  sunshine  etc. 

In  addition  to  displaying  the  presence  and  location  of  weather  phenomena  such  as  snow,  the 
icons  could  also  be  used  for  displaying  auxiliary  information. 

•  Tlie  density  of  the  displayed  icons  could  represent  the  magnitude  of  the  represented 
phenomena,  e.g.  the  number  of'snow  icons  per  unit  area  could  represent  the  amount 
of  snow  fall. 

•  The  color  or  texture  of  the  icon  could  display  the  predicted  probability  of  the  given 
phenomenon  (e.g.  rainfall). 

t  A  vector  attached  to  the  icon  could  display  the  rate  and  direction  of  movement  of  the 
given  phenomenon  (e.g.  storm), 

•  A  bar  chart  attached  to  the  icon  could  display  the  minimum,  maximum,  and  average 
for  the  given  phenomenon  such  as  rainfall,  at  the  given  time  of  year. 

5.3  Altitude  Variations 

Many  weather  phenomena  such  as  clouds,  temperature,  and  winds  vary  with  altitude.  These 
can  be  displayed  as  a  cross  sectional  average  in  the  altitude  band  of  interest,  by  using  colored 
overlay  areas.  A icematively,  they  can  be  viewed  as  three  dimensional  surfaces  which  can  be  used 
for  the  detailed  planning  of  flight  corridors.  In  particular,  when  weather  phenomena  interact 
with  mountainous  terrain  having  narrow  and  steep  passes  or  valleys,  it  is  very  necessary  to 
visualize  the  detailed  three  dimensional  flight  corridors. 

5.4  Distributed  Area  Displays 

For  some  weather  phenomena  such  as  wind,  temperature,  and  fog,  we  need  to  display  the 
spatial  variation  of  the  desired  phenomenon  over  a  relatively  large  area,  in  order  to  provide 


31 


information  needed  for  flight  planning  or  combat.  It  is  important  to  provide  this  display  as  a 
user  selectable  overlay,  and  also  to  ensure  that  displaying  weather  phenomena  does  not  obscure 
the  underlying  display  of  maps,  flight  plans,  threats,  or  mission  status.  This  constraint  can  be 
satisfied  by  a  careful  choice  of  the  colors  for  displaying  the  overlays,  and  by  associating 
transparency  with  the  selected  icons.  Another  way  is  to  represent  some  of  these  phenomena  by 
additively  modifying  the  texture  of  the  underlying  display. 

•  Wind  Patterns  can  be  displayed  by  using  streamlines. 

•  Temperature  Effects  can  be  displayed  by  using  contours  of  constant  temperature,  by 
color  coding,  or  by  texturing. 

•  Fog  can  be  displayed  by  using  contours  of  constant  fog  density,  by  color  coding,  or  by 
texturing. 

•  Daylight  can  be  displayed  by  using  contours  of  constant  light  level,  by  varying  color 
intensity  as  a  function  of  lighting  level,  or  by  texturing. 

5.5  Terrain  Effects 

Many  weather  phenomena  such  as  rainfall,  snowfall,  or  storms  are  modified  locally  by  the 
terrain.  Examples  include  shielding  by  a  mountain,  runoff  along  a  slope,  and  accumulation  or 
channelling  in  valleys.  Using  available  technology  in  conjunction  with  smart  maps,  it  is  possible 
to  compute  the  effect  of  terrain  on  weather  patterns;  and  to  incorporate  it  into  an  integrated 
display,  with  suitable  color  coding,  intensity  modulation,  transparency  /  opacity  effects, 
luminosity  and  illumination  effects,  and  texturing. 

5.6  Flight  Path  Attributes 

It  should  be  possible  to  augment  the  display  of  flight  paths  with  selected  attributes  of  the 
weather  such  as  visibility,  ceiling,  snow,  rain,  and  fog.  This  can  be  achieved  by  varying  the  color, 
width,  or  line  pattern  of  the  lines  representing  flight  paths.  It  is  important  that  the  displayed 
attributes  do  not  obscure  the  main  objective  of  displaying  a  flight  path:  viz.  showing  a  path 
between  the  origin  and  the  destination,  as  well  as  showing  way  points. 

5.7  Animation 

Animation  could  be  used  to  display  either  the  local  variation  of  weather  phenomena  over 
time,  or  to  chart  out  the  anticipated  movement  of  storms.  Such  animations,  when  coupled  to  an 
animation  of  mission  plans,  can  provide  veiy  useful  inputs  to  planners,  and  enable  them  to 
readily  detect  possible  conflicts  with  respect  to  predicted  weather  patterns. 
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5.8  Summary 

The  following  is  a  brief  summary  of  the  graphics  requirements  for  weather  representation. 

•  2D/3D  Pictorials 

•  Icons  to  represent  different  types  of  weather  (clouds,  rain,  snow) 

•  Density  of  icons  to  represent  magnitude  of  weather  phenomena 

•  Color  or  texture  of  icon  to  display  predicted  probability 

•  Vector  attached  to  icon  to  represent  direction  and  magnitude  or  rate  of  movement 

•  Bar  charts  to  show  ranges  of  weather  phenomena(currently,  past  history,  and 
predicted) 

•  Showing  2D  weather  patterns  over  an  area 
Streamlines  to  display  wind  patterns 

Color  coding  or  texturing  to  show  contours  of  temperature  or  fog  density 

Use  of  overlays  with  careful  selection  of  colors  and  transparency  to  show 
combinations  of  weather  phenomena  (wind,  snow) 

•  Weather  phenomena  such  as  winds  and  clouds  vary  with  altitude.  This  can  be 
displayed  either  as  cross-sectional  averages  in  a  band  of  altitudes  or  as  a  3D  shaded 
surface,  where  the  surface  represents  a  constant  weather  parameter. 

•  Weather  data  can  be  directly  related  to  the  individual  legs  of  flight  routes.  Visibility, 
snow,  rain,  and  wind  can  be  directly  associated  with  a  route  leg  by  varying  line  color, 
width,  or  line  pattern.  Specific  data  values  can  be  accessible  via  icons  or  pop-up 
menu  related  to  a  route. 

•  Modelling 

'lbrrain(e.g.,  mountains)  affects  weather  patterns.  It  is  possible  to  model  the  effect  of  terrain 
on  weather. 

•  Animation 

•  Display  the  projected  pattern  of  weather  movement  over  a  geographical  area  for  a 
mission.  The  time  period  for  the  weather  projection  must  correspond  to  the  time 
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period  prior  and  during  the  mission.  Overlay  weather  animation  and  flight  routing 
animation  to  show  potential  impact  of  weather. 
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6  Route  Planning 

6.1  Introduction 

Route  planning  is  a  component  of  both  force  level  and  unit  (or  squadron)  level  planning 
although  each  aspect  requires  different  levels  of  detail  and  information.  Routes  are  the  paths  of 
aircraft  as  th^*  travel  to  and  from  targets.  Routes  consist  of  two  parts:  legs  and  tumpoints. 
Tumpoints  are  locations  in  a  path  where  headings  change  while  legs  connect  two  tumpoints  and 
are  normally  depicted  as  straight  lines. 

The  goals  of  each  planning  level  differ  and  consequentfy,  their  requirements  for  route 
representation  differ.  Routes  are  used  to: 

•  Show  the  coordination  of  aircraft  dispatched  from  numerous  staging  bases.  This  may 
include  multiple  sorties  of  aircraft  over  a  period  of  time. 

•  Show  the  overall  flow  of  aircraft  in  relation  to  political  and  geographic  boundaries. 

•  Show  alternative  routes. 

•  Show  the  distances  and  fuel  consumption  between  tumpoints. 

•  Show  the  location  of  flight  paths  with  respect  to  known  threats,  targets,  front  lines, 
friendly  bases,  ground  features  and  terrain  contours. 

•  Show  time  constraints  of  the  aircraft  with  respect  to  dictated  times  on  target  (TOT). 

•  Show  both  attack  and  egress  routes. 

•  Show  the  survivability  of  routes. 

•  Show  the  effects  of  weather  ceilings  on  flight  paths. 

•  Show  those  portions  of  a  route  that  are  under  threat 

•  Show  the  feasibility  of  completing  a  mission  given  the  available  aircraft,  ordinance^  and 
squadron  combat  radius. 

This  section  assumes  that  software  exists  to  calculate  headings,  fuel  consumption,  speeds 
and  distance.  Also,  software  may  exist  that  can  automaticalty  generate  routes.  Here,  we 
concentrate  on  the  requirements  for  route  representations  as  th^  apply  to  both  force  and  unit 
level  planning. 

6.2  For<»  Level  Planning 

Force  level  planning  produces  an  air  tasking  order  (ATO)  that  is  used  as  a  basis  for  the  flight 
plans  generated  during  unit  level  planning.  At  the  force  level,  (X)ncetns  are  at  a  broad  level. 
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involving  coordination  of  multiple  resources  from,  possibly,  multiple  staging  bases.  Hence,  the 
force  level  planner  needs  to  visualize  the  many  relationships  between  many  active  participants. 
Routes  created  during  this  step  need  not  take  into  account  details  such  as  ground  features  or 
details  of  threats  or  weather.  Here  the  graphics  requirements  are  two  dimensional,  because"  of 
the  high  level  view  required. 

Force  level  concerns  are: 

•  Geographic  features  on  maps  can  overwhelm  a  graphics  screen  with  information.  What 

graphical  techniques  will  highlight  routes  on  a  busy  screen  covered  with  map  data? 

•  highlight  by  color  -  reserve  fully  saturated  colors  for  routes. 

•  highlight  by  thickness  -  draw  legs  as  thick  lines. 

•  highlight  by  blinking  -  under  user  control,  blink  routes.  If  something  changes  (like 
the  location  of  a  tumpoint)  about  a  route,  blink  it. 

•  highlight  by  animation  -  move  icons  along  routes.  Direction  and  speed  of  movement 
can  be  controlled  by  the  user. 

•  suppress  map  background  -  increase  the  white  level  of  the  map  background.  This 
causes  the  maps  colors  to  appear  more  pastel.  The  more  saturated  route  colors  will 
be  more  visible. 

•  Routes  are  paths  that  evolve  over  time.  What  graphical  techniques  show  temporal 

relationships? 

•  shape  animation  -  change  the  width  of  the  route,  making  it  wider  as  it  approaches 
the  target. 

•  lookup  table  animation  -  a  technique  used  in  TV  weather  broadcasts,  change  the 
texture  or  color  of  a  route  as  a  function  of  time. 

•  color  coding  of  time  -  map  time  onto  the  color  hue  circle,  going  from  blue,  cyan, 
green,  yellow,  to  red.  As  the  route  approaches  its  endpoint,  the  color  becomes  red. 


•  Routes  may  intersect  or  overlap  each  other  on  a  graphics  display.  How  do  we  show  the 
entrance  and  exit  of  an  intersection?  How  do  we  show  if  two  or  more  routes  overlap? 


•  alternate  display  -  alternate  the  display  of  each  route. 


36 


•  change  shape  -  use  different  shapes  for  each  route  with  a  legend  displayed  on  the 
screen. 

•  change  color  -  use  different  color  for  each  route,  perhaps  tying  the  color  codes  to 
the  colors  of  the  originating  staging  base  or  target. 

•  logical  combinations  -  when  routes  overlap,  use  logical  operations  to  create  a  new 
combined  color. 

•  Parameters  such  as  fuel  consumption  vary  over  the  course  of  a  route.  How  can  we  show 
such  data  on  a  route? 

•  map  data  to  shape  -  map  a  scaled  fuel  level  to  the  width  of  the  route. 

•  map  scalar  to  color  -  map  a  scaled  fuel  level  to  the  color  of  the  turnpoint  symbol. 

•  tie  gauges  to  turnpoints  -  fuel  gauge  can  be  attached  to  turnpoints. 

•  Paths  may  violate  some  political  or  practical  concerns.  How  do  we  show  these 
violations? 

•  highlight  via  sound  -  let  user  move  icon  along  route.  Voice  can  explain  any  violations 
as  icon  moves. 

•  highlight  by  flashing  -  those  portions  of  the  route  that  cause  violation  can  be 
flashed. 

•  highlight  by  shape  change  -  thicken  routes  where  they  violate  constraints. 
Superimpose  a  special  icon  over  those  parts  of  the  route  that  are  in  violation.  The 
icon  indicates  the  nature  of  the  violation. 

•  highlight  by  color  change  -  use  a  reserved  color  to  show  violations. 

•  Alternative  routes  may  be  generated  by  analysis  programs.  How  can  we  show  the 
differences  between  the  routes? 

•  simultaneous  animation  -  show  an  icon  moving  along  all  active  routes  to  see  how 
the  routes  relate  versus  time. 

•  sequential  animation  -  show  an  icon  moving  along  one  route. 

•  Routes  include  a  direction,  either  attack  or  egress.  How  can  we  show  the  differences 
between  the  routes? 

•  color  for  direction  -  show  attack  routes  in  one  color  and  egress  routes  in  a 
complementary  color. 
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arrows  for  direction  -  use  arrows  to  show  direction  of  movement. 


6.3  Unit  Level  Planning 

Unit  level  planning  takes  the  air  tasking  order  generated  at  the  force  level,  analyses  it  for 
feasibility,  creates  flight  plans  and  avionics  computer  input.  Since  a  route  created  during  force 
level  planning  is  not  necessarily  one  that  can  be  flown,  unit  level  planning  must  address  a  route’s 
feasibility  considering  current  aircraft,  personnel  and  ordinance  inventories.  At  this  level,  a 
planner  needs  to  visualize  routes  with  respect  to  more  detailed  topographic,  threat  and  weather 
representations,  but  three-dimensional  representations  are  not  required.  Contours  of 
elevations  and  weather  patterns  are  sufficient  for  the  unit  level  planning,  and  the  routes  can  be 
shown  superimposed  over  these  two-dimensional  representations  of  three-dimensional  data. 

Unit  level  issues  are: 

•  The  ATO  flight  plans  may  not  be  feasible.  How  do  we  highlight  the  bad  ones  and  which 

constraints  do  they  violate? 

•  highlight  by  blinking  -  blink  those  legs  of  a  route  that  are  in  violation. 

•  highlight  by  color  coding  -  color  violating  legs  with  a  specified  color. 

•  Weather  ceilings  may  change  flight  corridor.  How  do  we  show  ceilings  affect  on  flight 

path? 

•  side  view  of  weather  ceiling  -  show  a  vertical  cross  section  of  the  weather  pattern 
and  the  route  projected  onto  the  cross  section. 

•  color  coding  by  route  altitude  -  vary  the  color  of  the  route  according  to  its  altitude. 

•  color  coding  by  weather  parameters  -  color  the  route  using  weather  parameters 
such  as  pressure  and  wind  velocity. 

6.4  Mission  Rehearsal 

Pilots  use  route  plans  to  study  the  details  of  the  mission  including  the  topography  and 
geography  to  be  encountered  as  well  as  details  of  the  threats  that  can  be  expected  during  the 
mission.  From  the  pilot’s  point  of  view,  three-dimensional  representations  of  topography, 
threats  and  weather  are  critical  to  his  understanding  of  the  mission. 

Mission  Rehearsal  issues  are: 
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•  The  flight  path  of  an  airplane  is  three-dimensional.  What  techniques  are  available  to 
visualize  three-dimensional  paths? 

•  parametric  cubic  splines  -  represent  the  3D  path  using  parametric  cubic  splines, 
i.e.  one  spline  per  dimension  parameterized  by  chord  length. 

•  polygons  -  show  the  path  as  a  series  of  polygons,  coloring  the  top  and  bottom  sides 
of  the  polygons  differently  to  highlight  complex  maneuvers. 

•  other  primitives  -  use  cylinders  with  regular  polygon  cross  sections  to  show  twisting 
of  the  path. 

•  train  tracks  -  show  two  parallel  tracks  with  cross  polygons  as  is  done  in  the  Pilot’s 
Associates  video. 

•  A  target  protected  by  many  threats  may  have  limited  approach  paths.  These  paths  are 
inherently  three-dimensional.  How  can  we  represent  the  acceptable  flight  paths? 

•  volume  modeling  -  model  the  complement  of  the  threat  zones  using  volume 
modeling  techniques.  The  resulting  path  will  appear  as  geometric  volume  that  can 
be  interactively  navigated. 

•  multiple  views  -  show  top  and  side  two  dimensional  views  simultaneously. 

•  Alternative  legs  may  exits  at  a  tumpoint.  How  do  we  show  these  alternatives  and  their 
relative  effectiveness? 

•  legs  as  spokes  from  a  tumpoint  -  at  each  tumpoint  show  the  splitting  as  spokes  of 
a  wheel.  Let  the  user  highlight  a  selected  spoke  to  see  its  relation  to  alternatives. 

•  map  effectiveness  to  color  -  change  the  color  of  the  alternates,  mapping  probability 
of  success,  fuel  consumption,  deviation  from  time  on  target,  etc. 

•  Resources  such  as  fuel  and  ordinance  are  consumed  during  a  flight.  How  do  we  show 
the  status  of  these  resources  as  a  mission  unfolds? 

•  vary  width  of  path  -  change  the  width  of  the  path  according  to  fuel  and  ordinance 
reserves. 

•  vary  color  of  path  -  map  resources  to  the  color  of  the  path. 

•  TUmpoint  correlation  with  topographic  features  is  critical  during  the  flight.  How  do  we 
relate  recognizable  features  to  the  flight  path? 

•  panoramic  views  using  recon  data  -  using  optical  video  disk,  display  images  of  recon 
data  tied  to  individual  turnpoints  or  locations  on  a  leg. 
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»  3D  typography  tied  to  turnpoints  -  display  3D  topography  when  the  user  clicks  on 
a  tumpoint.  This  technique  uses  the  actual  path  as  an  active  icon. 

6.5  Summary 

Route  representation  plays  a  role  in  all  aspects  of  planning:  force  level,  unit  level  and 
mission  rehearsal.  As  planning  proceeds  from  the  big  picture  to  the  pilot’s  mission,  the 
graphical  requirements  move  from  two-dimensions  to  three-dimensions.  Color  and  animation 
play  key  roles  in  all  areas  of  route  planning. 

•  Overlays  -  to  make  routes  visible  on  maps,  make  the  map  colors  pastel.  When  routes 
overlap,  use  logical  “and”  and  “or”  to  show  the  combination  of  two  or  more  routes. 

•  Geometric  Representations  -  change  the  shape  of  routes  when  they  intersect  or  overlap. 
Map  scalar  data  such  as  fuel  reserves  to  the  shape  of  a  route.  Show  route  direction  with 
arrows.  Change  a  route’s  thickness  (make  it  bulge)  when  a  constraint  is  violated. 
Represent  3D  paths  with  parametric  cubic  splines.  Use  higher  order  primitives  such  as 
cylinders  and  train  tracks  to  show  twisting  3D  flight  paths. 

•  Modeling  -  create  geometric  models  of  flight  paths  using  volume  modeling  techniques. 
The  complement  of  threat  zone  volumes  will  show  a  safe  path. 

•  Color  -  use  fully  saturated  colors  for  routes  to  emphasis  them  on  maps.  Map  scalar 
values  such  as  time  to  target,  fuel  level,  and  probability  of  success  to  the  color  of  a  route. 
Use  color  to  distinguish  route  direction  (attack  or  egress).  Highlight  constraint 
violations  with  color. 

•  Animation  -  highlight  routes  on  a  busy  screen  by  moving  icons  along  the  route.  When 
routes  overlay,  alternate  the  display  of  each  route.  Compare  the  relationships  between 
multiple  paths  by  moving  icons  along  multiple  paths,  synchronizing  the  motion  with 
time. 

•  Data  Association  -  associate  fuel  gauges  with  turnpoints.  Associate  3D  topography  to 
turnpoints.  Associate  video  clips  of  recon  data  to  turnpoints. 

•  Multiple  Views  -  Show  top  and  side  views  of  topography  and  weather. 
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7  User  Interaction 


7.1  Introduction 

All  planning  activities  that  use  computers  require  special  attention  to  the  human-computer 
interaction.  This  interaction  includes: 

•  introducing  input  data  to  the  system  (constraints,  targets,  threats,  etc.), 

•  observing  the  results  of  intermediate  and  final  calculations, 

•  evaluating  alternate  solutions, 

•  storing  data  into  data  base, 

•  retrieving  data  from  data  base, 

•  receiving  update  information  relevant  to  planned  mission 

In  this  section  we  present  the  general  characteristics  and  requirements  of  a  good  user 
interface.  At  the  end  of  the  section  we  present  an  overview  of  windowing  environments  and  user 
interface  management  systems. 

7.2  User  Interface  Characteristics 

The  characteristics  of  a  good  user  interface  have  been  studied  in  many  papers  and  books 
dedicated  to  this  subject  [1].  Some  authors  provide  lengthy  lists  of  do’s  and  don’ts  in  user 
interface  design[18, 21].  These  suggestions  are  based  on  extensive  research  of  user  psychology 
and  statistics  gathered  from  various  case  studies.  A  good  user  interface: 

•  is  invisible,  e.g.  it  does  not  create  breaks  in  user  concentration  on  the  performed  tasks, 

•  provides  closure,  e.g.  user  is  aware  of  the  operation  performed  and  is  notified  when 
operation  is  done, 

•  requires  minimum  training  (simplicity), 

•  has  high  transfer-of-training, 

•  is  predictable  (consistency), 

•  is  easy  to  recover  from  errors, 

•  is  flexible. 
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•  is  intelligent, 

•  is  fast 

Users  know  right  away  if  they  are  happy  with  a  user  interface.  If  unhappy,  there  is  real 
danger  that  the  softv/are  won’t  get  used.  This  is  a  very  condensed  list  and  each  of  its  entries  can 
be  elaborated  into  pages  of  specifications;  for  example,  the  menu  and  button  selections  have  to 
be  structured  to  provide  convenient  and  fast  retrieval  of  information.  This  involves  many  issues 
of  choosing  the  right  color,  fonts,  and  selecting  an  appropriate  layout  for  control  buttons  and 
display  areas.  A  great  deal  of  experience  is  required  to  avoid  common  pitfalls  in  creating  a  good 
user  interface.  The  following  list  identifies  important  user  interface  requirements; 

•  Map  and  photograph  manipulation  This  includes  techniques  for  accessing  map  data 
or  data  base.  Selected  maps  should  be  displayed  and  user  should  be  able  to  zoom,  pan, 
or  select  different  layers  of  the  map.  Possible  selection  devices  include  pointers  (mouse, 
touch-screen,  digitizing  tablet,  and  light  pen)  or  menus  and  buttons. 

•  Manipulation  of  2D  graphics  data  Selection  and  dragging  of  2D  graphics  on  the  screen 
should  be  provided.  Some  2D  graphics  images  may  have  different  representation  modes 
that  should  be  selectable  by  the  user  from  menus  or  user  interface  buttons.  For  example, 
putting  a  lasso  around  a  threat  representation  and  moving  it  to  another  location,  or 
selecting  a  threat  via  a  mouse  pointer  or  a  pop-up  menu  and  removing  it  entirely  from 
map. 

•  Manipulation  of  3D  graphics  data  This  includes  techniques  for  manipulating  2D 
graphics  data  and  should  support  intuitive  techniques  to  do  translations  and  rotation 
in  3D  space.  Since  most  current  pointing  devices  only  support  2D  input,  special 
processing  is  required  to  do  3D  manipulation.  New  devices,  like  data  gloves,  are  true 
3D  input  devices.  Entertainment  games,  such  as  Nintendo,  use  data  gloves  to  remotely 
transmit  the  intention  of  the  user  to  participants  on  the  display  screen.  For  example, 
in  a  boxing  match,  a  movement  of  the  user’s  right  arm  in  a  certain  direction  and  speed 
will  be  directly  transmitted  and  used  by  one  of  the  boxers  on  the  screen.  In  an  Air  Force 
planning  application,  the  user  can  remotely  transmit  his  intent,  such  as  send  a  signal 
to  start  the  animation  of  a  plane(s)  route(s). 

•  Timely  notification  of  important  events  The  user  can  be  alerted  to  high-priority  changes 
in  the  display  using  a  number  of  techniques,  such  as  icon  changes,  changes  in  color  or 
patterns,  pop-up  windows,  audio  signals,  and  speech  synthesis.  Some 
application-  specific  uses  for  alerts  are  changes  in  weather,  threat  or  target  movements, 
or  changes  in  mission  requirements  or  resources. 
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•  Concurrent  Viewing  of  Information  Sources  Because  of  the  large  amount  and  variety 
of  tactical  planning  information,  and  the  need  to  understand  relationships  between  the 
types  of  information,  a  specific  requirement  is  to  be  able  to  view  different  data  sources 
or  alternate  plans  concurrently.  A  graphical  technique  for  satisfying  this  need  is  a 
windowing  system  such  as  X.  Since  this  is  an  important  graphics  requirement,  a 
subsequent  section  provides  more  detail  on  Windowing  Environments. 

•  Switching  between  alternate  sources  of  sensor  data  Different  data  sources  (e.g.,  radar, 
infrared,  or  visual  data)  can  be  represented  as  overlays  to  the  map  data.  Overlays  can 
be  selected  individually  or  in  combinations,  or  animated  by  cycling. 

•  Switching  between  different  representations  For  example,  ordnance  can  be  presented 
visually  as  bitmaps,  text,  or  small  icons. 

•  Animating  objects  in  the  display  Animation  is  extremely  useful  to  represent  changes 
in  space  and  time.  The  user  must  be  able  to  easily  interact  with  the  animation.  This 
interaction  involves  the  creation,  manipulation,  movement,  and  storage  of  animation 
objects.  Many  of  the  VCR-type  functions  must  be  available,  such  as  pause  and  rewind. 
Editing  of  the  animation  must  be  interactive,  e.g.,  changing  the  route  way  points  and 
reinitiating  the  animation.  A  remote  3D  device  such  as  a  data  glove  could  allow  a  pilot 
or  planner  to  control  a  flight  path  and  change  its  direction  as  he  reacted  to  threats  or 
weather  patterns  (similar  to  a  video  game). 

•  Customizing  the  user  interface  A  major  portion  of  application  software  development 
is  devoted  to  the  creation  of  user  interfaces.  Graphics  tools  should  be  provided  for  easy 
change  of  layout  and  functionality  of  user  interface  objects.  A  subsequent  section 
discusses  User  Interface  E>esign  Tools.  Since  there  are  so  many  combinations  of  overlays 
possible  (how  many  to  use,  whether  to  display  them  statically  or  cycle  through  them 
dynamically,  and  what  each  one  should  contain),  it  is  imperative  that  the  planner  be  able 
to  design  them  to  suit  his  needs. 

•  Simulation  of  virtual  environments  A  virtual  environment  is  a  computer  simulation  of 
some  physical  processes.  In  mission  rehearsal,  for  example,  the  computer  can  simulate 
a  fly-by  of  the  aircraft  through  a  3D  environment  along  the  planned  route,  and  the  pilot 
can  observe  the  scenery  on  the  computer  screen  or  with  a  head  mounted  display. 

7.3  Windowing  Environments 

The  large  amount  of  screen  real  estate  available  on  workstations  requires  software  to 
manage  it.  In  the  early  1970s,  researchers  at  Xerox  PARC  began  experimenting  with  systems  that 
allowed  the  user  to  partition  computer  displays  into  multiple  regions,  or  windows.  Coupled  with 
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the  invention  of  the  mouse  at  PARC  in  the  1960s,  this  made  possible  much  of  the  user  interface 
display  technology  available  on  today’s  workstations.  In  this  section  we  discuss  the  basic 
terminology  of  window  systems  and  the  two  most  likely  window  systems  that  will  be  in  use  under 
UNIX  for  the  next  five  years,  Xll  and  NeWS. 

7.3.1  Window  System  Terminology 

A  window  system  divides  the  display  into  multiple  regions,  usually  called  windows.  (They  are 
sometimes  called  canvases  or  layers.)  Wndows  are  organized  into  trees,  with  each  window 
having  a  single  parent.  A  v.'indow  is  clipped  to  the  boundaries  of  its  parent  window.  The  root 
window  provides  the  background  for  all  other  windows  displayed.  Top-level  windows  aie 
windows  whose  parent  is  the  root  window.  The  window  system  user  interface  (its  look  and  feel)  is 
governed  by  a  window  manager.  It  is  responsible  for  screen  management  of  all  top-level 
windows,  such  as: 

•  whether  or  not  windows  can  overlap, 

•  how  the  initial  location  of  a  new  window  is  set, 

•  how  menus  are  displayed, 

•  which  window  receives  characters  when  the  user  types  on  the  keyboard  (the  keyboard 
focus),  and 

•  how  the  keyboard  focus  is  moved  between  windows. 

Window  management  functionality  has  traditionally  been  part  of  the  window  system  itself. 
With  the  release  of  X  and  NeWS,  a  trend  has  emerged  to  have  window  managers  as  separate 
programs. 

7.3.2X11 

Most  early  window  systems  were  tied  to  the  hardware  and  operating  system  software  on 
which  they  ran,  so  it  was  virtually  impossible  to  write  portable  applications  that  made  use  of 
window  system  functionality.  The  X  Window  System,  developed  by  MIT’s  Project  Athena, 
changed  that.  Development  progressed  internal  to  MIT  until  X,  Version  10  was  made  generally 
available  to  the  public.  The  current  version,  Xll,  is  now  widely  available  from  many  computer 
vendors,  as  well  as  from  MIT  over  the  Internet.  Xll  is  a  network-based  window  system 
consisting  of  a  server  that  manages  display  resources  (frame  buffer,  mouse,  and  keyboard)  for 
client  applications  and  a  protocol  that  the  server  uses  to  communicate  with  its  clients.  The  two 
major  contributions  made  by  Xll  are: 
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•  Standardization  -  Many  computer  vendors  now  offer  X  as  the  only,  or  predominant, 
window  system  running  on  their  machines.  Coupled  with  graphics  standards,  such  as 
PHIGS,  and  with  standardized  user  interfaces,  such  as  MOTIF  or  Open  Look,  Xll 
holds  the  promise  of  portable  graphics  and  window-based  applications. 

•  X  protocol  -  The  server  and  its  clients  communicate  using  the  X protocol.  This  allows 
applications  running  on  (say)  a  supercomputer  to  display  results  on  the  workstation  on 
the  user’s  desk.  Better  use  is  made  of  both  resources,  the  supercomputer  and  the 
workstation. 

In  some  respects,  however,  Xll  is  like  older  window  systems.  For  instance,  it  is  pixel-based, 
so  an  image  displayed  on  two  different  monitors  will  look  different  if  their  resolutions  are 
different.  Device  resolution  independence  must  be  programmed  into  each  application. 

Xll  is  currently  the  standard  window  system  for  much  of  the  computer  industry,  and  will  be 
for  several  years  to  come.  Currently  the  easiest  way  to  develop  portable  applications  is  probably 
to  develop  them  to  run  under  Xll  using  a  publicly  available  toolkit,  such  as  the  HP  Widgets. 

7.3,3  News 

News,  from  Sun  Microsystems,  is  another  window  system,  similar  in  some  respects  to  X.  It 
is  network-based,  like  X,  so  server  and  client  can  reside  on  different  computers,  and  it  is 
inherently  policy-free;  window  management  functions  are  performed  by  a  separate  client 
program. 

However,  NeWS  is  strikingly  different  than  X  in  one  major  respect.  Instead  of  using  a 
special-purpose  protocol  to  communicate  between  server  and  client,  NeWS  uses  a  superset  of 
PostScript,  the  graphics  page  description  language  developed  by  Adobe,  This  makes  NeWS 
display  resolution-independent,  and  gives  the  software  developer  very  significant  gains  in 
expressive  power,  because  PostScript  was  designed  as  a  high-level  graphics  language. 

7.4  User  Interface  Design  Tools 

As  user  interfaces  have  become  more  sophisticated,  th^  have  become  harder  to  develop. 
Xll  application  developers  who  program  using  one  of  the  toolkits  available,  such  as  the  HP 
widgets,  have  three  subroutine  libraries  to  choose  from,  containing  700  or  more  functions.  Of 
those  700-plus  functions,  typically  between  50  and  150  will  be  needed  by  most  applications. 
Other  window  systems  have  programming  interfaces  of  similar  complexity. 

This  complexity  leads  to  low  quality  user  interfaces.  While  some  initial  design  of  the  user 
interface  may  be  done,  the  difficulty  inherent  in  modifying  the  interface  means  that  for  all 
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practical  purposes,  the  first  interface  that  is  functionally  correct  (the  one  with  the  correct 
number  of  buttons  and  windows)  is  the  final  one.  Even  simple  tasks,  such  as  lining  up  a  row  of 
buttons  or  changing  object  colors,  is  often  not  performed,  because  it  is  too  tedious  a  task.  Little 
or  no  user  interface  design  modification  is  attempted  for  most  window-based  applications. 

Graphical  user  interface  editors  are  now  available  from  a  number  of  sources.  They  promise 
to  revolutionize  the  way  graphical  user  interfaces  are  designed  and  to  dramatically  reduce  the 
time  necessary  to  develop  a  mature  user  interface.  Some  tools,  such  as  VAPS  and  Data  Views, 
are  quite  mature,  and  are  geared  toward  specific  application  domains.  Others,  such  as  UIMX, 
GUIDE,  and  TAE  Plus,  are  more  general,  but  less  mature. 

User  interface  layout  tools  are  important  because  the  user  interface,  as  discussed  above, 
must  be  flexible.  The  planner  should  be  able  to  modify  many  properties  of  the  user  interface 
without  resorting  to  traditional  programming  methods.  Much  of  the  flexibility  necessary  can  be 
achieved  using  layout  tools.  For  other  tasks,  special  tools  must  be  developed,  however,  many  of 
the  prop>erties  they  must  exhibit  are  those  of  good  layout  tools. 

7.5  3D  Input 

Three-dimensional  input  is  a  cumbersome  task,  because  most  displays  and  input  devices 
are  two-dimensional.  No  single  solution  allows  the  user  to  manipulate  objects  in  the  dimension 
perpendicular  to  the  display  device.  Moreover,  in  the  context  of  the  tactical  Air  Force 
envirorunent,  3D  input  is  not  generally  required  except  in  those  instances  where  the  third 
dimension  of  altitude  is  important.  This  is  often  the  case  for  pilots  who  plan  the  details  of  their 
route.  However,  it  would  be  best  to  use  the  usual  aircraft  controls  in  order  to  navigate  this  space, 
since  this  is  what  the  pilot  is  familiar  with. 

When  3D  input  is  needed  by  planners,  some  echniques  have  been  developed  to  cope  with 
this  problem.  Chen  [14]  discusses  several  schemes  for  rotation  of  3D  objects  using  a  mouse.  In 
particular,  one  technique  he  calls  virtual  spheres  works  quite  well.  In  this  paradigm,  the  object  to 
be  manipulated  is  located  in  the  center  of  an  imaginary  sphere,  and  the  user  clicks  on  the  sphere 
with  a  mouse,  rotating  the  sphere  (and  thus  the  object  inside  into  a  new  position).  Nielson  and 
Olsen  [28]  describe  a  technique  for  3D  manipulation  (rotation,  scaling  and  translation)  that 
assigns  motion  along  or  about  the  three  Cartesian  axes  based  upon  the  relationship  between 
their  projection  on  the  display  and  the  movement  vector  of  the  mouse.  Additional  possibilities 
for  3D  input  include  the  space  ball  and  the  data  glove. 

7.6  Summary 

The  following  is  a  brief  summary  of  graphical  requirements  for  user  interaction. 
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•  2D  Input 

•  Selection  and  dragging  of  2D  graphic  representations:  mouse,  touch-screen, 
digitizing  tablet,  menus,  buttons.  Ability  to  pan  and  zoom. 

•  3D  Input 

•  Ability  to  perform  translations  and  rotations  in  space. 

•  3D  input  devices  such  as  data  gloves  to  interact  with  objects  on  the  display 

•  2D/3D  Pictorials 

•  User  notification  of  important  events:  icon  changes,  pop-up  windows,  changes  in 
color  or  pattern,  audio,  speech  synthesis 

•  Multiple  Displays 

•  \fiewing  of  concurrent  sources  of  information,  such  as  weather,  threat,  and 
animations:  windowing  systems  and  techniques  such  as  Hypermedia 

•  Overlays/Groupings 

•  Ease  of  interaction  with  overlays  of  different  data  sources,  e.g.,  sensors 

•  Switch  between  different  representations:  2D  map  with  icons,  3D  shaded  polygons, 
photographs,  text 

•  Animation 

•  Ease  of  interaction  in  creating,  manipulating,  moving,  and  storing  animation  objects 

•  Editing  animation 

•  3D  remote  interaction  with  animation,  such  as  data  glove 

•  Interactive  Displays 

•  Customizing  user  interface:  design  tools  for  user  interface 

•  \firtual  Environment 

•  Immerse  user  in  a  realistic  simulation  of  his  environment,  e.g.,  mission  rehearsal. 
Graphic  methods:  animation,  head-mounted  displays,  3D  input  devices 
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MISSION 


ROME  LABORATORY 

Rome  Laboratory  plans  and  executes  an  interdisciplinary  program  in  re¬ 
search,  development,  test,  and  technology  transition  in  support  of  Air 

0 

Force  Command,  Control,  Communications  and  Intelligence  (C  I)  activities 
for  all  Air  Force  platforms.  It  also  executes  selected  acquisition  programs 
in  several  areas  of  expertise.  Technical  and  engineering  support  vfithin 
areas  of  competence  is  provided  to  BSD  Program  Offices  (POs)  and  other 

O 

BSD  elements  to  perform  effective  acquisition  of  C  l  systems.  In  addition, 
Rome  Lahoratorys  technology  supports  other  AFSC  Product  Divisions,  the 
Air  Force  user  community,  and  other  DOD  and  non-DOD  agencies.  Rome 
Laboratory  maintains  technical  competence  and  research  programs  in  areas 
including,  but  not  limited  to,  communications,  command  and  control,  battle 
management,  intelligence  information  processing,  computational  sciences 
and  software  producibility,  wide  area  surveillance/sensors,  signal  proces¬ 
sing,  solid  state  sciences,  photonics,  electromagnetic  technology,  super¬ 
conductivity,  and  electronic  reliability/mcdntainability  and  testability. 


